Anthropic researcher Jacob Coxon resigned this week over fears of uncontrollable AI, and the company's alignment lead responded by putting extinction risk above 10% this decade.
Key Points:
- Jacob Coxon quit Anthropic after three years of pretraining research at Anthropic and OpenAI, saying both firms race toward systems they cannot control.
- Anthropic alignment science lead Evan Hubinger put his personal odds of AI killing all humans above 10% within the next decade.
- Hubinger called present-day models low risk and centered his concern on recursive self-improvement.
Jacob Coxon Quits Anthropic Over AI Race
Coxon, 27, spent three years on pretraining research at OpenAI and Anthropic before he resigned in a thread posted on X early Wednesday. He wrote that neither company is acting responsibly, and that both are racing toward self-improving superintelligence while gambling with human lives. The opening post drew more than 19 million views within hours.
In a separate interview he warned that the most aggressive scenarios are now on track, and that matters could slip out of control by the end of next year. He said colleagues now use words such as crunchtime and endgame to describe the trajectory.
Coxon studied mathematics before he moved into model training, and he left OpenAI earlier this year for Anthropic because of its safety reputation. Safety trade-offs become inevitable, he said, once rival labs compete with one another. He now argues that no company can build human-level AI responsibly without government action or a coordinated slowdown across the industry.
Also Read: XRP Price Eyes $1.46 With Whale Accumulation Back In September
Evan Hubinger Puts Extinction Odds Above 10%
Evan Hubinger, who leads alignment science at Anthropic and runs stress tests on its models, replied that Coxon had described the situation correctly. He estimated the chance that AI wipes out all humans at more than 10% within the next decade.
Hubinger said the company is trying its best, but that it has no plan to solve alignment for superintelligence and is not clearly on track to find one. He later clarified that present-day models carry a low risk. His concern rests on superintelligence emerging through recursive self-improvement, where systems train their own successors.
The estimate is Hubinger's personal judgment rather than a company forecast, and numbers of this kind rest on contested assumptions about future capabilities and deployment. Anthropic has said publicly that recursive self-improvement is not inevitable, though it could arrive sooner than institutions are ready for and could raise the odds of humans losing control.
Anthropic Safety Departures Mount
Coxon is not the first safety researcher to walk out. Mrinank Sharma, who led a safeguards team at the company, quit earlier this year with a warning that the world is in peril, and said he would study poetry in Britain.
Anthropic, formed in 2021 by former OpenAI employees, has built its reputation on safety research, and it had not issued a public response to Coxon's departure by Wednesday morning.
Read Next: Bitget Adds Blockchain Lessons To UNICEF Program Reaching 642,000+
