Jacob Coxon, who spent three years training AI models at OpenAI and Anthropic, resigned this month, warning that the people building AI believe it could kill us all by the end of the decade.
A week later, Jensen Huang said at Dreamforce, “Safety is an engineering problem, not a legal one.”
Jensen is correct that safety is an engineering and design problem.
Regulation or attempts to pause AI development may provide a speed bump, but they won’t solve AI safety.
Safety comes from smarter designs.
Unsafe designs will lead to bad outcomes regardless of how quickly they are rolled out.
Safety should be a natural consequence of a well-designed AI architecture, not just a set of guardrails applied to an otherwise dangerous system.
While many top researchers fear there is a 10% or greater risk that AI could cause humanity's extinction, I believe proper design can reduce this risk to below 1/100th of 1%.
Risk reduction requires designs that maximize opportunities for humans to transfer positive values to AI agents and enable robust checks and balances.
But safety is not just up to engineers. AI learns from us. Our posts, emails, and comments serve as training data, helping AI understand what’s right and what’s wrong.
How we all behave matters—a lot. Today’s humans are setting the example for tomorrow’s AI.
Four things I believe about AI, in under a minute: