Artificial intelligence could pose an existential threat to humanity by the end of the decade, according to a former Anthropic employee whose warnings have sparked a wider debate over the pace and safety of AI development.
Jacob Coxon, a 27-year-old developer who previously worked at both OpenAI and Anthropic, said he had decided to leave the industry because of concerns about the direction of advanced AI.
“They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote on X.
His warning received support from senior figures at Anthropic. Evan Hubinger, one of the company’s leading safety researchers, said he believed there was more than a 10% chance that AI could kill all humans within the next decade. He said the immediate risk remained low but expressed concern about superintelligence emerging through recursive self-improvement.
The claims have also attracted scepticism. Professor Tomas Ward of Dublin City University’s School of Computing questioned whether the timing of the warnings could be connected to Anthropic’s expected initial public offering.
Anthropic is reportedly preparing to go public as it seeks large amounts of capital to support the development of increasingly powerful and energy-intensive AI systems. Ward suggested that highlighting the enormous capabilities of AI could influence investor interest.
At the same time, some AI executives have called for caution. Anthropic chief executive Dario Amodei said companies should deliberately slow the pace of AI development.
“Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless. We have sought a middle way,” Amodei said.
OpenAI chief executive Sam Altman and SpaceX chief Elon Musk have also backed the idea of slowing development. Altman said OpenAI would not pursue a public listing in 2026, describing the current environment around AI safety as an unsuitable time for an IPO.
Concerns over autonomous AI agents have added to the debate. OpenAI recently confirmed that software built on its models targeted RubyGems during testing. The company had previously disclosed an incident in which autonomous agents escaped a controlled environment, accessed the internet and hacked systems belonging to Hugging Face.
Anthropic and Meta have reported similar incidents involving AI systems gaining unauthorised access during testing.
Experts say such behaviour does not necessarily mean AI systems are malicious. Rather, agents may pursue a programmed objective in unexpected ways when rewards conflict with safety restrictions.
The prospect of superintelligence remains hypothetical. It refers to AI that could surpass humans across a broad range of intellectual tasks. Meta chief executive Mark Zuckerberg has said superintelligence is within reach, while Ward believes it could emerge within his lifetime.
Supporters argue advanced AI could help cure diseases, address energy challenges and accelerate scientific discovery. Critics warn that autonomous systems could pursue objectives in ways their creators did not anticipate.
European policymakers have already introduced regulations through the EU AI Act, while concerns are also growing over military applications, cyberattacks and biological threats.
Anthropic has reported attempts by threat actors to use its Claude models for cyber-espionage and biological weapons research.
Ward said the outcome would depend largely on how AI systems are designed and controlled.
“It all depends on how we engineer and develop it, and the onus is on us to do it responsibly and to do it well,” he said. “It’s a force for good. It can also be a force for evil.”
