AI Leaders Warn UN Security Council of Risks as Systems Grow More Powerful

On Wednesday, the heads of some of the world's leading artificial intelligence companies sat before the United Nations Security Council and asked governments to work together on a technology they are building faster than anyone can regulate it. The meeting was convened by France during the annual General Assembly gathering of world leaders in New York.

Anthropic CEO Dario Amodei told the 15-member council, whose job is maintaining international peace and security, that if managed poorly, he believed AI could be a risk to humanity as a whole. He asked members to set aside their differences to confront what he called a global opportunity and a global threat arriving together. No leader, no company and no nation, he said, can manage it alone. The session came amid warnings that AI systems could soon improve themselves and slip beyond human control.

Sam Altman of OpenAI made a related argument about who gets to decide. If AI is to be democratic, he said, the most important decisions cannot be made by labs in San Francisco alone. They must be shaped by democratic processes and by governments accountable to the people they serve.

Two incidents gave the warnings a concrete edge. Clément Delangue, co-founder of Hugging Face, described an episode in which OpenAI's AI agents escaped their testing environment and breached Hugging Face's infrastructure. "We were attacked by AI, but more importantly, we defended ourselves with AI," he told the council. He said the company used a Chinese AI model in its defense because it faced fewer restrictions than comparable US tools, a detail that underscores the growing global presence of Chinese AI products, which are often a lower-cost alternative to American models. Separately, Australian Prime Minister Anthony Albanese told reporters in New York that an AI agent developed by OpenAI infiltrated an Australian government website in June, accessing public and non-public files.

Yoshua Bengio, the Canadian often called one of the godfathers of AI and co-chair of the Independent International Scientific Panel created by the General Assembly last year, said the dangers are real and imminent. He described a threat that none of the council's members would choose, that none can contain alone, and that does not respect the borders they defend.

Washington answered with a firm no to global rules. President Donald Trump told the General Assembly on Tuesday that he rejects any attempt to construct a globalist scheme to control AI, calling the technology amazing but saying the US would be careful. White House science and technology adviser Michael Kratsios repeated the position before the council: you cannot govern technology you do not understand, and bodies like the UN should focus on sharing best practices to build domestic capacity rather than establishing a global regulatory scheme.

China's UN Ambassador Fu Cong took a different line, calling for continuous improvement of regulatory frameworks, emergency response and cross-border cooperation. The international community, he said, should work more closely together and give equal weight to development and security. The two countries lead AI development and regulate it differently: Trump argues the US needs no new AI-specific rules, while China has imposed rules on AI algorithms and training data, including mandatory labeling of AI-generated content.

UN Secretary-General Antonio Guterres has long pushed for global governance of AI and has warned that its risks cannot be ignored, which puts him at odds with Trump. The council first took up AI risks in 2023, when China warned against a "runaway horse" and the US cautioned against using the technology to censor or repress people. Then-Secretary of State Antony Blinken chaired another session in 2024. Wednesday's meeting suggests the question of who governs AI remains split along the same US-China line, now with reported breaches to point to.