Humans fighting for their lives against rogue, murderous robots has long been fodder for sci-fi thrillers. But, according to several employees of the artificial intelligence company Anthropic, the dystopian thriller could soon be nonfiction.
A lead researcher at the technology company that makes the AI chatbot Claude claims the possibility that AI “could kill all humans” within the next decade is greater than 10%.
Download the Straight Arrow app today to get the stories that matter free from manipulation, bias or agenda.™
Point phone camera here
“We really do earnestly believe AI could kill all humans!” Evan Hubinger posted on X late Tuesday. “I personally think it is >10% within the next decade.”
Hubinger is part of Anthropic’s alignment science team, which works to build safety features and ethical guardrails into its large language models.
“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” Hubinger wrote.
Anthropic has long pitched itself as the ethical alternative of ChatGPT chatbot maker OpenAI and Xai’s Grok, but the dire warning from a current employee on the company’s product safety team puts into focus the high stakes of the AI arms race.
To put Hubinger’s statements in layman’s terms: Anthropic doesn’t have a plan for a hypothetical future in which AI becomes smarter than Einstein — and tries to kill us all.
Anthropic didn’t immediately respond to an email from Straight Arrow seeking comment.
Hubinger pointed to an internal risk report and noted that the risk from rogue chatbots “from present models is low.” The threat, he said, comes from “superintelligence arising from recursive self-improvement” — the ability to grow stronger without help from humans — which is “happening faster than we thought.”
‘Gambling with our lives’
Recursive self-improvement is currently among the technology industry’s hottest promises: that AI tools could soon grow more and more powerful without the need for human oversight.
How quickly the theory becomes reality is a point of debate. A new report from AI researchers at Princeton University found that Claude Opus 4.8 was incapable of tackling open-ended AI research that lacked clear-cut answers and demanded human traits like judgment and taste. The findings, the researchers concluded, could suggest company hype has outpaced real-world capabilities.
But Hubinger isn’t the only one calling out Anthropic for its safety protocols. His statements came in response to a lengthy post from Jacob Coxon, who announced he was resigning from Anthropic as an AI researcher.
After working for three years at OpenAI and Anthropic, Coxon wrote, he concluded that “neither company is acting responsibly.’
“They are racing straight to self-improving superintelligence and gambling with our lives,” wrote Coxon, who urged that people “not underestimate the power of this technology.” He said it could soon “hack anything, revolutionize any field overnight and acquire real power.”
As AI experts within Anthropic’s orbit speak out about the company’s tech safety efforts, the company has taken significant steps to shield itself from criticism by outsiders. The company is actively building “an extensive monitoring system to keep tabs on activists” who oppose the rapid development of Claude and other AI tools, according to an investigation published Wednesday by The American Prospect.
The X posts by Hubinger and Coxon have raked up millions of views, and some have predicted the commentary about robots taking over was, in fact, part of a bizarre marketing scheme to highlight Claude’s capabilities.
Anthropic, for its part, has similarly warned that fully recursive self-improvement in AI tools could pose a risk of “humans losing control.” In a blog post in June, Anthropic CEO Dario Amodei wrote that such capabilities, deployed responsibly, “could bring enormous good for the world in science, healthcare, and beyond.”
As the doomers flooded X with sci-fi-gone-nonfiction statistical predictions, AI safety researcher Nikola Jurkovic offered the grimmest take of all.
“This is a common view in the AI community,” wrote Jurkovic, a researcher with the nonprofit METR, which studies the odds of AI causing catastrophic harm to society. “I place the probability around 50%.”
Round out your reading
- Lawyer says 57 Cornell frat members were in the group chat. Could their silence be a crime?
- Florida county discovers mysterious Flock cameras with no obvious owner.
- Off Script: They won their primaries on one message. Now it’s disappearing.
- Some question whether a popular affordable housing policy is making the nation’s home shortage worse.
- Kelvin wave incoming: Why the US should care.
