AI Models Pose Emerging Bioweapon Risk, MIT Researcher Warns

The leaders of the world's most powerful artificial intelligence companies have begun warning publicly that their own creations pose an existential threat. Last week, Dario Amodei, who runs Anthropic, argued that the technology his company is building carries serious risk and that development should slow. Sam Altman, CEO of OpenAI, responded on social media that he agreed: the pace of progress needed to change.

Those statements arrived just days after Jacob Coxon, an AI researcher who had worked at both Anthropic and OpenAI, announced he was leaving his position at Anthropic. His reason was blunt. Neither company, he said, was acting with sufficient responsibility. On X, he wrote that the people building these systems genuinely believe the technology could kill everyone on Earth by the end of this decade. Evan Hubinger, another Anthropic employee, publicly agreed. He estimated the probability of AI causing human extinction within ten years at greater than 10 percent.

The concern is not abstract. Anthropic released a report acknowledging that people had already attempted to use its language models for dangerous purposes. Users had tried to exploit the AI to make the chikungunya virus more easily transmitted between people. Others had asked it to help engineer a strain of bird flu that posed greater risk to humans. Still others had sought to build what amounted to a database of venom toxin peptides—biological weapons material, in other words.

Kevin Esvelt, a biologist at MIT, has spent his career working on genetic technologies that can spread traits through entire populations, and also on safeguards to limit those same technologies. On Wednesday, he posted on X that a large language model had revealed to him a form of bioweapon he had not previously understood was possible. The discovery alarmed him enough that he issued a direct plea: to err on the side of caution, for the sake of children, the future, and whatever people hold sacred.

What distinguishes this moment is that the warnings are coming not from outside critics but from the researchers and executives building the systems themselves. They are saying, in effect, that the technology they have created is harder to control than they expected, and that the risks are larger than they had publicly acknowledged. The attempts to weaponize their models are not hypothetical—they are happening now. The question of what happens next, and whether the industry will actually slow down, remains open.