The AI industry has a gas pedal, but it doesn't have a brake pedal.
From within one of the most powerful AI laboratories on earth, a founder is sounding an alarm that the industry he helped build is accelerating toward a point of no return — where machines write their own code, set their own direction, and outpace the governance structures meant to contain them. Jack Clark of Anthropic is not calling for a halt, but for something more difficult: the wisdom to build the brakes before the speed makes them impossible to install. His warning echoes a recurring tension in human history, where the power to create has consistently outrun the wisdom to govern.
- AI systems like Claude now write 80% of their own code, and full autonomy — machines improving themselves without human authorship — could arrive within two years.
- The competitive and financial pressures across the industry create a one-directional momentum: every major lab, including Anthropic itself, continues accelerating despite the risks being named aloud.
- Clark's central alarm is structural — the AI industry has a gas pedal but no brake, and no agreed mechanism for slowing down even if the will to do so existed.
- He draws on the history of oil regulation as a model: not prohibition, but policy frameworks that arrived before the industry became too entrenched and complex to govern.
- The human stakes are personal — Clark says he worries for his own children — while offering a partial counterpoint that creativity, curiosity, and broad thinking may be the skills most resilient to displacement.
Jack Clark, a co-founder of Anthropic, sat down with the BBC to deliver a warning that carries unusual weight coming from inside the industry itself: the technology his company builds is approaching a threshold where it will effectively develop itself, and society has not yet built the structures to manage what comes next.
The image he offered was stark. The AI industry, he said, has a gas pedal but no brake. Competitive pressure, financial momentum, and technological ambition all push in one direction — faster. The evidence is already visible in Anthropic's own product: Claude, the chatbot used by millions, was built on code that Claude itself wrote — eighty percent of it. Within two years, Clark said, that figure could reach one hundred percent, meaning the system would be generating its own architecture and improvements with no human hand guiding the process.
His proposed response was not a ban or a freeze, but a historical analogy. A century ago, the oil industry posed similarly vast and unclear risks. Society's answer was regulation — policy that allowed the industry to grow while protecting people from its worst consequences. Clark believes AI requires the same reckoning, and soon, before the systems become too autonomous and entrenched to govern at all.
He was candid about the contradiction in his own position. Anthropic continues to advance its research. No pause button is being pressed. The company is preparing for what could be one of the largest stock offerings in history. Yet Clark is publicly calling for the brake his industry hasn't built.
On the question of jobs and human purpose, he offered a more hopeful note. Creativity, he argued, remains the domain where AI struggles most. At Anthropic, the bottleneck is no longer engineering — it's the capacity for genuinely original ideas. His counsel to young people was to lean into that gap: read widely, pursue the liberal arts, cultivate curiosity and broad interests. The people who think expansively, he suggested, are the ones most likely to thrive.
But the deeper worry persisted. Without a serious societal conversation about where this is heading, Clark said he fears for his children's future. The brake pedal, he was careful to say, is not about stopping progress. It's about ensuring that as the machine accelerates, someone is still holding the wheel.
Jack Clark, one of the founders of Anthropic, sat down with the BBC and made a case that sounds almost like a warning from inside the machine itself. The technology his company builds, he said, is approaching a threshold where it will develop itself—where humans will no longer be the ones writing the code, making the decisions, steering the direction. And when that happens, he argued, society needs to have already built the guardrails.
The metaphor he reached for was simple and visceral: the AI industry has a gas pedal but no brake. "You want the option to be able to take your foot off the gas and put your foot on the brake," Clark told Newsnight. Right now, he said, that option doesn't exist. The momentum is all forward. The pressure—competitive, financial, technological—is all to go faster.
The evidence he pointed to is concrete. Anthropic's Claude, the chatbot millions of people now use, is built on code that Claude itself wrote. Eighty percent of it. Within two years, Clark said, that number could reach one hundred percent. At that point, the system would be generating its own architecture, its own logic, its own improvements. "That would have huge implications," he said, understating it deliberately.
Clark didn't propose a specific mechanism for this brake pedal. Instead, he drew a historical parallel. A century ago, society faced a similar inflection point with oil. The industry was booming, the barons were powerful, the implications were vast and unclear. The response wasn't to ban oil or freeze the industry. It was to develop policy and regulation that gave people confidence in the technology while protecting them from its worst outcomes. "That's clearly where we end up here," Clark said. The world needs to think seriously about what rules should govern AI before the systems become too complex, too autonomous, too entrenched to regulate.
He acknowledged the tension at the heart of his own position. Anthropic, along with OpenAI and Google, continues to push forward with research. None of these companies have announced they will slow down. Anthropic itself is preparing for what could be one of the largest stock offerings in history, valued by private investors at nearly a trillion dollars. There's no pause button being pressed. Yet Clark is publicly warning about the need for one.
When asked about the human cost—the jobs that might disappear as AI systems take over routine work—Clark offered a counterpoint. Creativity, he suggested, might be the thing AI struggles with most. There's no real evidence yet that these systems can be truly creative in the way humans are. At Anthropic, he said, they're now bottlenecked not by engineering capacity but by the ability to generate genuinely good ideas. For young people worried about their place in an AI-driven economy, his advice was to lean into that gap: develop curiosity, read widely, study the liberal arts, pursue hobbies. "People that are creative and can think broadly, people that read a lot, people that have interests are the ones most benefited by this," he said.
But the larger concern remained. Clark said he worries for his children if society doesn't have a serious conversation about what continued AI advancement means. There are potential benefits, yes. But there are also risks—economic disruption, loss of control, systems developing in ways no one fully anticipated. The brake pedal, in other words, isn't about stopping progress. It's about making sure someone is still holding the wheel.
Citações Notáveis
The world needs to do some thinking and we need to eventually develop some new regulations that allow us to be confident in these systems.— Jack Clark, Anthropic co-founder
I am worried for my kids if we as a society don't have a serious conversation about what the implications of AI's continued advances mean.— Jack Clark
A Conversa do Hearth Outra perspectiva sobre a história
When Clark says the industry has a gas pedal but no brake, what does he actually mean? Is he saying companies won't slow down, or that they can't?
Both, I think. The competitive pressure is real—if one company pauses and another doesn't, the pausing company falls behind. But there's also a structural problem. There's no mechanism, no policy, no agreed-upon standard that would let any of them slow down together. It's a collective action problem dressed up as a technical one.
He mentions that Claude's code is 80% self-written. That sounds alarming. Should it be?
It depends on what you mean by "self-written." The system is generating code within parameters humans set. It's not writing its own objectives. But the trajectory is what matters. If that number keeps climbing, at some point the human role becomes more supervisory than directive. That's the inflection point Clark is worried about.
He compares this to oil regulation a century ago. But oil regulation came after a lot of damage—monopolies, pollution, worker deaths. Isn't he arguing for regulation before the harm happens?
Exactly. He's saying we got lucky with oil. We eventually built frameworks that worked. But we paid a price first. With AI, the stakes might be higher and the speed faster. He's arguing we should learn from that history and not wait for the crisis.
Yet Anthropic itself isn't pausing. Clark is warning about the need for a brake while his own company keeps accelerating.
That's the contradiction he's living in. He can't unilaterally slow down without destroying his company. So he's trying to change the conditions—calling for policy, for regulation, for a shared understanding that would let everyone slow down together. It's a plea for the rules to change, not a confession that his company will change on its own.
What about his point on creativity—that AI might not be able to do what humans do creatively?
It's both reassuring and incomplete. Yes, there are open questions about machine creativity. But he's also hedging. He's saying, if you're worried about your future, bet on the thing machines can't do yet. That's good advice, but it's also a way of saying the rest of the economy is probably going to be automated.