Anthropic Wants a Global Pause on AI Development Before Humans Lose Control
Anthropic, the company behind the Claude chatbot, is calling on the world’s top artificial intelligence labs to work together on a coordinated global pause on AI development, warning that the technology is advancing faster than society can safely manage.
In a blog post published Thursday, Anthropic co-founder Jack Clark and Marina Favaro, head of its research institute, said it would be “good for the world to have the option to slow or temporarily pause” the development of frontier AI models. The post cited internal data and public benchmarks showing AI is already speeding up its own development, with Anthropic engineers now shipping roughly eight times more code per quarter compared to the 2021 to 2025 period.
The core concern is what researchers call “recursive self-improvement.” Based on current trends and sufficient computing power, an AI system could eventually be capable of designing and building its own successor without direct human intervention. Anthropic acknowledged this could deliver major breakthroughs in science and healthcare, but warned it “also might increase the risks of humans losing control over AI systems.”
The proposed global pause on AI development would not simply be a slowdown. Anthropic envisions a verifiable coordination mechanism that allows leading labs to confirm that rivals have genuinely stopped or reduced work, ensuring no bad actor exploits a pause to pull ahead in secret. The company argues that without such a structure, any unilateral slowdown would only benefit the “least cautious” players in the field.
Anthropic rival OpenAI pushed back with a different position in a report published Wednesday, arguing that “democratic governments not private companies acting alone must ultimately determine the rules, safeguards, and accountability mechanisms” governing AI development.
The call for a global pause on AI development comes at a complicated moment for Anthropic. The company has confidentially filed for an IPO that could value it at close to a trillion dollars, and its annualized revenue run rate is on track to hit $50 billion by the end of this month. Critics, including Trump adviser and venture capitalist David Sacks, have accused the company of pushing a “regulatory capture agenda” designed to hold back competitors under the cover of safety concerns.
The timing also follows a separate warning from University of Toronto researchers who demonstrated how AI tools can be used to create a self-adapting AI “worm” capable of spreading across networks and adjusting its hacking strategy as it goes. Lead researcher Nicolas Papernot noted that the threat is not limited to the most powerful models, saying anything connected to the internet is now at risk as the cost of mounting AI-powered cyberattacks continues to fall.
Regulation, meanwhile, has struggled to keep pace. A Trump administration executive order issued earlier this week places the burden on labs themselves, asking them to voluntarily submit their most capable models for government cybersecurity testing ahead of public release. It is a far cry from the binding international framework Anthropic says is now urgently needed.
