In the past week, the conversation around AI has rapidly moved beyond the capabilities of the most advanced models to concerns that AI could potentially cause human deaths within the next few years. Anthropic CEO Dario Amodei has now alleged that AI companies have downplayed or misrepresented the risks of the technology for years. He acknowledged that AI does carry genuine dangers.
Speaking to CBS, Amodei discussed concerns surrounding advanced AI systems and the risks that cannot simply be ignored. “I won't lie to you, there are real dangers. And I think for too long the industry lied to people about the fact that this technology had risk,” he said.
End of humanity is biggest risk of AI, says Amodei
According to the Anthropic CEO, one of the most serious dangers posed by AI could be the complete extinction of humanity. “The biggest risk could be the end of humanity,” Amodei said.
Amodei’s remarks came after former Anthropic researcher Jacob Coxon warned on X that companies such as Anthropic and OpenAI were effectively gambling with human lives. He said that people developing AI genuinely believe the technology could potentially wipe out humanity by the end of the decade.
Responding to Coxon’s warning, Amodei said he had not “fully just appreciated what it would actually be like when the progress was as fast as it was.” However, he stressed that the current speed of AI advancement should be viewed as a warning rather than a reason for panic. “It's a warning sign. It's a warning sign that we need to slow down,” he said.
In another interview with CNN, Amodei said he agreed with Coxon “much more than” he disagreed, although he stopped short of suggesting that a catastrophic outcome was inevitable. Coxon is not alone in raising concerns about AI potentially ending humanity. OpenAI safety researcher Marcus Williams recently suggested that there could be a 70 per cent probability of AI causing human extinction within the next three years.
Amodei has also pushed back against assigning a straightforward percentage to the risk, arguing that the outcome will depend on how the AI industry develops the technology. “If we build in the right way, I think the probability of something bad happening is very low,” he said. “If we build in the wrong way, the probability of something bad happening is very high.” His call for slowing the pace of AI development has received support from other industry leaders, with OpenAI CEO Sam Altman saying his company would also commit to independent evaluators being given employee-level access, while Elon Musk posted on X: “Dario is right.”
Anthropic won’t stop releasing new AI models
Such warnings may raise questions about whether AI companies such as Anthropic could stop releasing new models altogether. However, Amodei does not believe that should happen. Instead, he said every new generation of models must undergo appropriate safety testing before being released. “What it means is that we need to make sure that every generation of models that we release is properly tested,” he said.
In an essay posted on X, Amodei said Anthropic would “provide third-party evaluators with permanent, employee-level access” to its systems. This would allow them to examine model behaviour, training methods and whether the company was adhering to its stated safety commitments.
Amodei is not the only person proposing measures to monitor and regulate AI development. Ideas currently being discussed include banning superintelligence and requiring AI systems to have a mandatory “kill switch” capable of shutting down potentially dangerous models. Amodei said such a kill switch “could be a good idea”, but opposed a complete ban on superintelligence.
“One, the idea that we would never get the incredible benefits, would never get the medical cures, that doesn't sit right with me. Second, this technology will be built in other countries, including authoritarian countries,” he explained.
Asked whether the US should slow down AI development even if China chose not to, Amodei described it as the most difficult part of the situation. “This is the toughest dilemma. This is the toughest aspect of our situation,” he said.
On the issue of who should ultimately control AI technology, Amodei said he had always found it unusual that such powerful technology was being developed by a private company and admitted that he was uncomfortable with that arrangement. He said he would be open to oversight from “the right combination of governments”, while also cautioning that a single government could misuse AI technology just as easily as a single private company could.
