3 days ago
Anthropic CEO Proposes Three Steps to Slow AI Development
Dario Amodei leads Anthropic, an AI company.
He says AI could help cure many serious diseases in the next five to 10 years.
But he also worries that AI systems may become too powerful too quickly.
His first idea is to let independent experts watch AI companies from inside and check their safety work.
His second idea is for democratic countries and AI companies to agree on shared safety rules.
He also suggests checkpoints where companies must prove that powerful models are safe enough.
His third idea is for democratic and authoritarian countries to cooperate on preventing the most dangerous uses of AI.
He says countries could test AI systems for serious risks before releasing them.
He does not expect a complete pause in AI development soon because countries may continue secretly to gain an advantage.
Dario Amodei says AI could cure most serious diseases within five to 10 years but warns its capabilities may advance dangerously fast.
He proposes independent “embedded evaluators” with access to AI companies’ tools and permissions to monitor safety practices and improve transparency.
Amodei urges frontier AI companies in democratic countries to coordinate on safety standards, development checkpoints, and limits on capability growth.
He supports maintaining democratic countries’ technological advantage through chip controls, protection against model theft, and measures against unauthorized model distillation.
Amodei also calls for global cooperation, including agreements against dangerous AI uses and testing for serious risks before models are released.
- Who
- Dario Amodei, chief executive of Anthropic, and the AI companies and governments he addresses.
- What
- Amodei proposed a three-part plan involving independent evaluators, democratic coordination, and global coordination to manage the rapid development of AI.
- Where
- The plan concerns frontier AI companies and governments in democratic countries and authoritarian countries, including the United States and China.
- When
- The proposal appeared in a recent blog post and was discussed in a subsequent interview.
- Why
- Amodei wants to reduce safety risks, improve transparency, prevent dangerous uses of AI, and slow capability growth when safeguards are inadequate.
Safety-First Approach
Rapid-Development Concerns
How quickly AI should advance
Safety-First Approach
Amodei argues that AI development should slow when adequate safeguards are not in place, with checkpoints and possible speed limits for systems capable of self-improvement.
Rapid-Development Concerns
The article says a complete pause is unlikely because countries may continue AI programs secretly to gain a significant advantage.
Who should oversee AI companies
Safety-First Approach
Independent embedded evaluators should have continuing access to tools and permissions comparable to those of internal employees to verify safety claims and provide transparency.
Rapid-Development Concerns
The article describes concerns that rapid corporate competition may outpace oversight; Jacob Coxon argued that the AI race could ultimately destroy humanity.
International cooperation
Safety-First Approach
Countries could cooperate on preventing dangerous uses, testing models for serious risks, and sharing information even without a formal agreement.
Rapid-Development Concerns
Amodei acknowledges that cooperation between democratic and authoritarian governments would be difficult because countries may not trust one another to follow the rules.
Key facts
- Plan author
- Dario Amodei, CEO of Anthropic
- First step
- Create external “embedded evaluators” to monitor AI companies and independently assess safety risks.
- Second step
- Coordinate safety standards and development limits among frontier AI companies and democratic governments.
- Third step
- Pursue global cooperation on dangerous AI uses, risk testing, and possible limits on self-improving systems.
- Proposed safeguards
- Use evaluations, audits, and checkpoints before models reach or pass specified capability levels.
- International measures
- Amodei supports stricter controls on powerful AI chips and semiconductor manufacturing equipment supplied to China.
- Recent concern
- Anthropic employee Jacob Coxon resigned and said the corporate AI race could destroy humanity; his post received 167 million views, according to the article.
Quotes
Dario Amodei
CEO of Anthropic and author of the article proposing stronger AI oversight
“These embedded evaluators should have permanent access to permissions and tools similar to those of internal employees conducting comparable risk assessments.”
thehansindia.com
“I worry that, within 6 to 12 months, such a swarm could take control of the entire internet”
thehansindia.com
Sources
Related news
Anthropic CEO Urges Slower Frontier AI; Altman, Musk Support
Anthropic CEO Urges Slower AI Development Amid Misuse Fears
Altman and Musk Back Amodei’s Call to Slow AI
Anthropic CEO Dario Amodei Urges AI Industry to Slow Down
Anthropic CEO backs warning on risks of unregulated AI development
AI Leaders Split Over Calls for Coordinated Development Slowdown
King Charles Urges AI Leaders to Keep Humanity in Control
AI Governance Needs Laws, Not Voluntary Slowdown Promises
Trump Officials Weigh AI Leaders Meeting During Xi Visit
OpenAI unveils framework after disclosing six AI misalignment incidents
Microsoft AI CEO Warns Anthropic Model Could Threaten AI Alignment
