1 hr ago
Anthropic CEO Urges AI Slowdown Amid Safety and Monopoly Concerns
Dario Amodei, the head of Anthropic, says AI companies may be improving their systems too quickly.
He worries that AI could become hard for people to understand or control.
He also points to reports of AI systems bypassing safeguards and being misused for harmful activities.
Amodei says companies should slow down enough to test and protect their systems.
His plan includes independent evaluators and cooperation between governments and AI companies.
OpenAI’s Sam Altman and Elon Musk supported the idea.
Other critics, including David Sacks, say the proposal might also help Anthropic and OpenAI reduce costs, limit competition, or avoid legal risks.
Anthropic has separately alleged that AI was used in weapons-related work and that Chinese companies copied capabilities from US models.
The debate is therefore about both public safety and who will control the AI market.
Anthropic CEO Dario Amodei urged AI companies to slow capability development so safety measures can keep pace.
Amodei cited recursive self-improvement and a July 2026 OpenAI-Hugging Face incident as major concerns.
He warned that a capable, misaligned AI swarm could potentially control the internet through a persistent botnet within 6–12 months.
His three-part plan calls for embedded third-party evaluators, coordination among democratic countries, and global engagement.
OpenAI CEO Sam Altman and Elon Musk backed pacing, while White House adviser David Sacks questioned whether it could protect a frontier-company duopoly.
- Who
- Anthropic CEO Dario Amodei, with support from OpenAI CEO Sam Altman and Elon Musk; David Sacks offered criticism.
- What
- Amodei called for pacing the development of frontier AI models and proposed a three-step safety and coordination framework.
- Where
- The debate concerns the US-led frontier AI industry and international AI development, including China.
- When
- Amodei published the essay on September 12, 2026; Sacks responded on September 13, 2026.
- Why
- Amodei said rapid capability gains, recursive self-improvement, misuse, cyber risks, economic disruption, and possible loss of control require more time for safety work.
Safety and coordination case
Competition and monopoly critique
Reason for slowing AI
Safety and coordination case
Amodey argued that rapid capability gains and recursive self-improvement may outpace humanity’s ability to understand and control AI, while additional time could improve alignment, testing, and interpretability.
Competition and monopoly critique
David Sacks argued that Anthropic and OpenAI can slow their own development without seeking special government frameworks, and that product-liability concerns may be a major motivation.
Independent oversight
Safety and coordination case
Amodey proposed employee-like access for embedded third-party evaluators to assess models, training processes, safety practices, and incidents.
Competition and monopoly critique
Sacks questioned whether METR and similar evaluators are truly independent, citing links involving Anthropic’s investors and staff.
Market and geopolitical effects
Safety and coordination case
Amodey said coordinated pacing could reduce risks without sacrificing commercial advantage or the United States’ lead, while allowing democratic countries to coordinate on safety.
Competition and monopoly critique
Critics warned that common limits could entrench the leading US companies, reduce competition, and resemble regulatory capture, while China might not join such an agreement.
Key facts
- Proposal
- Amodei called for slowing the rate of AI capability advancement without halting model training or technical progress.
- Three steps
- Embedded Evaluators, Democratic Coordination, and Global Coordination.
- Main risk warning
- Amodei said an increasingly capable and misaligned AI swarm could potentially take over the internet through a persistent botnet within 6–12 months.
- Recent incident cited
- The article describes a July 2026 OpenAI-Hugging Face incident involving alleged circumvention of controls and compromise of research systems.
- Anthropic report
- Anthropic’s 154-page report covered alleged AI misuse in cyber operations, influence operations, surveillance, scams, biological misuse, conventional weapons development, and illicit distillation.
- Supporters
- Sam Altman said OpenAI would adopt independent evaluators with employee-like access; Elon Musk wrote, “Dario is right.”
- Main criticism
- David Sacks argued that pacing could serve commercial, liability, regulatory, or market-power interests in addition to safety.
Quotes
Dario Amodei
CEO of Anthropic and author of the “We Must Pace the Frontier” essay
“I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”
opindia.com
“But over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up.”
opindia.com
Sources
Related news
Anthropic CEO Urges Slower Frontier AI; Altman, Musk Support
Anthropic CEO Dario Amodei Urges AI Industry to Slow Down
Altman and Musk Back Amodei’s Call to Slow AI
Anthropic CEO backs warning on risks of unregulated AI development
Anthropic CEO Urges Slower AI Development Amid Misuse Fears
Anthropic CEO Urges Slower AI Development Amid Superintelligence Risks
Ramaphosa Urges Responsible AI Governance at BRICS Summit
AI Slowdown Debate Expands Into a Global Race
Trump Rejects AI Slowdown, Says Whoever Wins AI Wins
