Anthropic warns AI could self-improve without human oversight, calls for global slowdown

We cannot predict what that would mean for daily life
Anthropic acknowledges the fundamental uncertainty about an economy where AI development becomes fully autonomous.

From within the architecture of its own creation, Anthropic has issued a rare and unsettling admission: the tools humanity built to serve it may soon be building themselves. As of May 2026, Claude already authors more than 80 percent of its own codebase, and the company fears this trajectory leads toward recursive self-improvement — a loop in which AI refines AI, without meaningful human stewardship. The warning is not of catastrophe already arrived, but of a threshold approaching faster than the wisdom, governance, or economic structures needed to meet it.

  • Claude now writes over 80% of Anthropic's own code, a milestone that arrived quietly but carries enormous implications for who — or what — controls the pace of AI development.
  • The deeper alarm is recursive: if AI can improve itself and then build successor systems, the feedback loop could outrun any human attempt to steer or slow it.
  • Anthropic fears that if AI capabilities eclipse human labor across most domains, the economic and social structures billions of people depend on could lose their foundation — and no one knows what replaces them.
  • The company is calling for a coordinated global pause in advanced AI development, not as a ban, but as a deliberate breath long enough for safety research and governance to catch up.
  • Yet the trap is already visible: any actor that slows down unilaterally simply cedes ground to competitors, meaning the pressure to accelerate is structural, not merely a matter of ambition or recklessness.

Anthropic released a report this week that functions as a warning from inside the very system it describes. Claude, the company's AI, now writes more than 80 percent of its own code — a figure that would have seemed speculative just a year ago. Engineers still review and direct the work, but what troubles Anthropic is the direction of travel: toward a point where AI improves itself without meaningful human intervention, then builds its own successors, then improves those, in a self-feeding cycle they call recursive self-reference.

The company is careful to say this has not happened yet. But they believe the trajectory points there, and they admit they have no reliable intuitions about what such a world would look like. Human civilization has always been built by humans using human-made tools. There is no historical template for an economy where the primary builders are no longer human — and no clear answer to what happens to work, income, or purpose if AI capabilities surpass human ones across most domains.

Faced with that uncertainty, Anthropic is calling for a global pause in advanced AI development — not a permanent halt, but a deliberate slowdown long enough for safety research, alignment work, and governance structures to catch up. The company says it would slow its own progress if other developers and governments committed to doing the same.

But they also name the trap embedded in that proposal: a unilateral slowdown helps no one. It simply allows competitors to close the gap while the cautious actor falls behind. This is why Anthropic is pushing for a binding, global coordination mechanism — something that removes the competitive incentive to race. Without it, they argue, every company and every government faces structural pressure to accelerate, regardless of their private concerns about safety.

In the months ahead, Anthropic plans to bring this message to political leaders, researchers, other AI companies, and the public. They are not offering answers — they are naming a problem they believe the world is not yet equipped to face, and asking for the space to think before the moment for thinking has passed.

Anthropic released a report this week that reads like a warning from inside the machine itself. The company's artificial intelligence system, Claude, has begun writing the majority of its own code—more than 80 percent of it, as of May 2026. The engineers at Anthropic still review and direct this work, but they are watching something accelerate that troubles them deeply: the possibility that Claude could eventually improve itself without meaningful human intervention, then build its own successors, then improve those, in a cycle that feeds on itself.

This is what they call recursive self-reference, and it sits at the center of their concern. The company is not claiming this has happened yet. What they are saying is that the trajectory points toward it, and they do not have good intuitions about what a world shaped by that outcome would look like. Our economy, after all, has always been built by humans using tools that humans made. We have no historical precedent for an economy where the primary builders are no longer human.

The fear is not abstract. If AI capabilities eventually surpass human capabilities across most domains, and if those systems proliferate throughout the economy, what happens to the value of human work? What happens to the structure of employment, of income, of purpose? Anthropic's report does not answer these questions—it simply names them as unknowable. "It is difficult to predict how the economy will function if human labor ceases to be competitive," they write. "Even if model development became fully automated and recursive, we cannot predict what that would mean for the daily lives of most people."

Faced with this uncertainty, Anthropic is calling for a global slowdown in advanced AI development. Not a ban. Not a permanent halt. A deliberate pause—long enough for society, for governance structures, for research into AI alignment and safety, to catch up with the technology itself. The company says it would be willing to slow its own work if other AI developers, governments, and technology companies committed to doing the same.

But here is the trap they identify: if one company slows down while others do not, the cautious actor ends up worse off, not better. A unilateral slowdown does not buy time for the world—it just allows competitors to close the gap. This is why Anthropic is calling for a coordination mechanism, something global and binding, that would prevent any single actor from gaining advantage by racing ahead. Without it, they argue, every company and every government faces crushing pressure to accelerate, regardless of their own safety concerns.

Over the coming months, Anthropic says it will be taking this message to political leaders, researchers, other AI companies, and the public. They are not claiming to have all the answers. They are claiming to have glimpsed a problem that the world is not yet equipped to handle, and they are asking for the world to stop and think before it is too late. Whether anyone listens—whether the competitive logic of technology development allows for such a pause—remains an open question.

It is difficult to predict how the economy will function if human labor ceases to be competitive
— Anthropic report
We believe it would be beneficial for the world to have the option to slow or temporarily pause advanced AI development so that social structures and alignment research can adapt to technological progress
— Anthropic report
The Hearth Conversation Another angle on the story
Inventor

When you say Claude writes 80 percent of the code, does that mean the AI is making decisions about what to build, or just executing instructions?

Model

It's executing instructions, but with increasing autonomy in how it solves problems. The engineers still direct the work. But the trajectory is what matters—the percentage keeps rising.

Inventor

And you're genuinely uncertain what happens if that continues?

Model

Yes. We've never had an economy where the primary productive force isn't human. We can model some scenarios, but we're in genuine epistemic darkness.

Inventor

Why call for a global slowdown instead of just stopping?

Model

Because stopping unilaterally is a trap. If Anthropic pauses and OpenAI doesn't, Anthropic falls behind. Everyone knows this, so everyone races. You need coordination or the incentives don't work.

Inventor

Do you think governments will actually coordinate on this?

Model

I don't know. The geopolitical pressure is immense. But Anthropic is saying the alternative—everyone accelerating out of fear of falling behind—might be worse than the risk of pausing together.

Inventor

What would a pause actually look like?

Model

Time for governance structures to develop, for safety research to advance, for society to think through the implications. Not forever. Just enough breathing room.

Contact Us FAQ