{"id":2135640,"date":"2026-09-09T10:42:41","date_gmt":"2026-09-09T07:42:41","guid":{"rendered":"https:\/\/analyse.optim.biz\/?p=2135640"},"modified":"2026-09-09T10:42:41","modified_gmt":"2026-09-09T07:42:41","slug":"anthropic-safety-researcher-says-theres-a-more-than-10-chance-ai-could-kill-all-humans-within-the-next-decade","status":"publish","type":"post","link":"https:\/\/analyse.optim.biz\/?p=2135640","title":{"rendered":"Anthropic Safety Researcher Says There’s a More Than 10% Chance AI ‘Could Kill All Humans’ Within the Next Decade"},"content":{"rendered":"
\n
\n
\n

An Anthropic safety researcher has said there is a greater than 10% chance that AI \u201ccould kill all humans\u201d within the next decade, and admitted there is no coordinated plan to prevent it from happening.<\/p>\n<\/p>\n

U.S. company Anthropic, headed up by Dario Amodei, develops the Large Language Model (LLM) Claude and conducts AI research. Yesterday, one of the company\u2019s researchers, Jacob Coxon, announced his resignation on X \/ Twitter<\/a>, accusing both ChatGPT maker OpenAI and Anthropic of \u201cracing straight to self-improving superintelligence and gambling with our lives.\u201d<\/p>\n

\u201cNeither company is acting responsible,\u201d Coxon, who previously worked at OpenAI, said. \u201cDo not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources,\u201d he continued. \u201cWe have all witnessed the progress in each of these domains, and progress is not slowing.<\/p>\n

\u201cThe people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible \u2014 but I hear the same people express fear privately. No other human activity poses this level of danger.\u201d<\/p>\n

\"null\"<\/a>
OpenAI founder Sam Altman recently called the Hugging Face incident “a legitimate AI safety accident and alignment failure.” Photo by Sean Rayford\/Getty Images.<\/figcaption><\/figure>\n

Coxon said that staff at OpenAI, headed up by Sam Altman, \u201chave not deeply internalized the civilizational stakes,\u201d which is why work to rapidly improve its AI models continues despite the risks. \u201cAt Anthropic, the stakes are well-understood, but they are locked in a race to get there first,” he insisted. \u201cThey believe no one else will act responsibly, so they must do it themselves, despite the risk.\u201d<\/p>\n

\u201cAccepting this race and entering the \u2018endgame\u2019 is a hubristic gamble that should not be launched from a private company\u2019s Slack. Attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available.\u201d<\/p>\n

Coxon went on to say he was optimistic about the potential for coordination, but doubted AI companies were on track to prevent a global race, \u201cwhich may require costly actions such as a temporary ban on improving model capabilities.\u201d<\/p>\n

He ended his statement by urging AI lab researchers to consider what they were doing. \u201cDo you want to kick off a superintelligent RL run without a rigorous understanding of its mind?\u201d he asked. \u201cShould you put your head down because \u2018it\u2019s happening anyway\u2019 – or take this moment to call for different conditions?\u201d<\/p>\n

\"null\"<\/a>
Anthropic boss Dario Amodei has issued various AI warnings in the past. Photographer: Prakash Singh\/Bloomberg via Getty Images.<\/figcaption><\/figure>\n

Coxon\u2019s statement has sent shockwaves throughout social media, sparking alarm and a string of mainstream headlines. But then it was backed up by Evan Hubinger, Anthropic\u2019s \u2018Alignment Science\u2019 lead, who added an even starker warning.<\/p>\n

\u201cJacob is correct here \u2014 we really do earnestly believe AI could kill all humans!\u201d he said<\/a>. \u201cI personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.\u201d<\/p>\n

Remarkably, Hubinger pointed to Anthropic\u2019s latest Risk Report, in which the company said it believes the risk from present models is \u201clow,\u201d rather than suggest a pause that might help prevent the extinction of humankind. \u201cWhat I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought,\u201d he added.<\/p>\n

The comments come after the Financial Times reported<\/u><\/a> that Anthropic withheld its latest model from the UK’s AI Safety Institute, one of the leading bodies in the world for assessing AI risk.<\/p>\n

This summer has seen a number of worrying developments that show how AI has become hard to control, even by those who create it. OpenAI, Anthropic, and Meta have all disclosed hacks carried out by their autonomous AI agents. OpenAI chief scientist Jakub Pachocki recently called for “extreme caution” over AI progress, warning more intervention may be needed to ensure “humans remain in control of the future.\u201d<\/p>\n

\"null\"<\/a>
U.S. President Donald Trump has backed unrestricted development of AI. Photo by Ludovic MARIN \/ AFP via Getty Images.<\/figcaption><\/figure>\n

However, the U.S. government has shown no sign of stepping in to force restrictions upon or even suggest a slowing down of AI development. In fact, quite the opposite. Earlier this month, the Trump administration backed OpenAI in \u2060a lawsuit against the New York Times, arguing in favor of the use of copyrighted writing to train artificial intelligence. \u201cThe United States has a strong interest in continuing to develop a robust and competitive artificial intelligence industry that sets the standard for the practice and procedure of AI use globally\u2026 As such, it is critical for the United States to \u2018retain global leadership in artificial intelligence,’\u201d the government’s brief said, referencing an executive order<\/a> President Donald Trump signed last year.<\/p>\n

Meanwhile, despite dire warnings coming from staff at Anthropic and OpenAI, neither company seems willing to pull the plug. Coxon and Hubinger\u2019s statements have sparked extreme concern across social media, with some expressing disbelief that work to improve AI is continuing even with such significant risk.<\/p>\n

\u201cHey! Here\u2019s a novel idea\u2026 could we just f***ing not kill ourselves due to an insatiable appetite for power and money?\u201d one person said. \u201cCrazy take: maybe you shouldn\u2019t be allowed to gamble with our lives?\u201d another added. \u201cIf I literally claimed that there was a 10% chance that I was building something that would kill people, the FBI would have me arrested. How are you not in jail?\u201d another asked.<\/p>\n

Photo Illustration by Dominika Zarzycka\/SOPA Images\/LightRocket via Getty Images.<\/em><\/p>\n

<\/div>\n
<\/p>\n
\"Wesley<\/div>\n

<\/p>\n

Wesley Yin-Poole<\/div>\n

<\/a><\/p>\n

Wesley is Director, News at IGN. He’s obsessed with fighting games, sports games and first-person shooters. His favourite game of all time is Street Fighter 2.<\/div>\n

<\/p>\n