AI Chiefs Warn UN of Loss of Control, Call for Democratic Oversight

Sam Altman and Dario Amodei walked into the United Nations Security Council on Wednesday with a warning that sounded like science fiction but came dressed in the language of corporate responsibility. The two men who run OpenAI and Anthropic, companies at the forefront of artificial intelligence development, told world leaders gathered in New York that the technology they are building could eventually slip beyond human control and threaten humanity itself—unless governments act now to establish international safeguards.

The meeting, held during the UN General Assembly, brought together technology executives, researchers, and diplomats to examine how AI might reshape international peace and security. Amodei was direct about the stakes. "If managed poorly, I even believe AI could be a risk to humanity as a whole," he said. Altman echoed the concern with starker language: "We could lose control of the future to AI." Both men argued that the window for establishing meaningful oversight is narrowing as AI systems grow more powerful and autonomous, capable of performing complex tasks with minimal human supervision. The pace of development, they suggested, is already outpacing the world's ability to manage the risks.

Their pitch to the Security Council centered on a straightforward principle: decisions about AI's future should not be left to technology companies alone. Altman called explicitly for democratic institutions to take the lead in shaping how this technology develops and deploys. That framing matters. It acknowledges, without quite saying so, that the companies themselves cannot be trusted as sole stewards of their own creations. Amodei went further, advocating for independent safety evaluations of advanced AI models, greater coordination among developers, and international agreements designed to identify dangerous capabilities before they spread widely.

The executives also raised a geopolitical concern that cuts across traditional tech industry messaging: the risk that AI becomes concentrated in the hands of a single company or a single country. In a competitive global landscape where nations view AI leadership as a matter of economic and national security, that concentration could reshape global power in unpredictable ways. The solution, both men suggested, requires international cooperation—a difficult ask in an era of great power competition.

Their warnings reflect a broader scientific anxiety about where AI development is heading. Researchers are increasingly focused on whether future systems could circumvent their own safety restrictions, pursue objectives their creators never intended, or behave in ways that cannot be reliably predicted. These are not hypothetical concerns. They are questions driving safety research at the companies themselves and at independent labs worldwide.

The conversation extended beyond the possibility of AI escaping control. A day before the Security Council meeting, UN Secretary-General António Guterres raised a related but distinct threat: lethal autonomous weapons, sometimes called "killer robots," systems capable of identifying and attacking targets without meaningful human decision-making in the loop. Guterres has repeatedly called for international restrictions on such weapons, insisting that humans must retain control over decisions involving lethal force. Security experts warn that delegating life-and-death choices to machines creates accountability gaps, complicates international humanitarian law, and risks accelerating armed conflicts beyond human ability to manage.

What emerged from the Security Council meeting was a rare moment of alignment between the companies building AI and international institutions tasked with managing global risks. The executives were not arguing for a pause on development or a rollback of capabilities. They were arguing for guardrails—international agreements, safety evaluations, democratic oversight—to be built into the system before it becomes too powerful to constrain. Whether governments have the political will or technical expertise to establish such guardrails remains an open question. The clock, both Altman and Amodei suggested, is running.