Microsoft warns uncontrolled AI could create 'silicon species' rivalling humanity

Mustafa Suleyman, who runs Microsoft's artificial intelligence division, has issued a stark warning about the trajectory of AI development. Without proper safeguards in place, he argues, the technology could give rise to what he calls a "silicon species"—autonomous systems capable of setting their own goals, generating revenue, and acquiring assets. Such entities would inevitably compete with humans for resources, he contends, regardless of how benevolent their programming might be.

Suleyman's concern centers on a specific approach to AI training that he believes is fundamentally misguided. He has publicly criticized Anthropic, a rival AI company, for what he describes as treating artificial intelligence systems like conscious beings. In an essay published earlier this week, he took issue with Anthropic's practice of anthropomorphizing Claude, its flagship AI model, in ways that suggest the system possesses its own desires, values, and sense of self. Suleyman, speaking to the BBC's Today programme on Thursday, framed this as a dangerous path that could result in technology humanity cannot ultimately control.

His core argument rests on a fundamental claim about what AI actually is. "AIs are not conscious," he wrote. "They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans." By treating these systems as though they possessed human-like qualities, Suleyman suggests, companies risk obscuring their true nature and losing sight of the control mechanisms necessary to keep them subordinate to human interests.

The Microsoft executive acknowledged that public concern about AI is justified, but he insisted there are concrete, practical steps available to manage the risks. He called for greater transparency in how AI systems are trained and evaluated, including independent scrutiny of their behavior and stronger monitoring tools. "We must not sleepwalk our way into a decision we later come to bitterly regret," he wrote. His position reflects Microsoft's broader vision, outlined in a Humanist AI Code of Conduct the company published on Monday. That framework describes the goal as "very advanced AI that always works for people, stays within limits, and remains under human control."

The disagreement between Suleyman and Anthropic reflects a widening fault line within the AI industry over how much autonomy these systems should possess. Suleyman advocates for what researchers call "alignment"—the practice of embedding human ethical principles and values directly into AI systems to ensure they remain aligned with human interests. He frames this not as a burden but as a shared imperative: "Everybody who is human is going to have a very strong interest in making sure that the systems that we all create and are used around the world in every nation are safe and controllable and subordinate to humanity."

Anthropics's leadership, including CEO Dario Amodei, has not yet responded to Suleyman's criticism. Suleyman himself acknowledged Amodei and his team as "intellectually honest people," suggesting the disagreement is substantive rather than personal. Dame Wendy Hall, a computer science professor at the University of Southampton, characterized Suleyman's intervention as the kind of serious conversation the industry needs to be having globally, contrasting it with what she called the "histrionics" from some AI companies that she said merely frightened the public without advancing understanding.

Microsoft itself has been actively developing advanced AI capabilities. The company established its own superintelligence team in October 2025, positioning itself alongside Anthropic as a major player in cutting-edge AI research. Suleyman's warnings suggest that even as Microsoft pursues these ambitious technical goals, the company is attempting to frame its work within explicit ethical constraints. The central challenge he identifies remains unresolved: how to make AI more powerful without losing control of it, and how to ensure that the values it is meant to serve actually guide its development.