2 hrs ago
Jacob Coxon Urges AI Kill Switches Amid Slowdown Debate
Jacob Coxon is worried that very powerful AI systems could become dangerous.
He says AI companies should slow down while they can still control their systems.
One idea is a kill switch that could turn an AI system off.
Coxon said this could work for many systems today.
However, large data centers would need many switches.
He warned that a switch might not work if an AI system spread across the internet and hacked computers.
Coxon also wants AI companies to create their own safety rules.
Several technology leaders support slowing AI development.
Satya Nadella has instead indicated support for safety evaluators built into AI systems.
Former Anthropic researcher Jacob Coxon backed calls to slow artificial intelligence development and proposed kill switches.
Coxon said kill switches could work for many current AI systems, though large data centers would require numerous switches.
He warned that kill switches might fail if future AI systems spread across the internet and conduct hacking operations.
Coxon urged AI companies to regulate themselves and create a framework for slowing development.
Dario Amodei, Sam Altman, Elon Musk and Demis Hassabis supported slowing development, while Satya Nadella backed embedded AI evaluators.
- Who
- Former Anthropic researcher Jacob Coxon, along with technology leaders including Dario Amodei, Sam Altman, Elon Musk, Demis Hassabis and Satya Nadella.
- What
- Coxon advocated slowing AI development, considering kill switches and encouraging self-regulation because of risks from increasingly powerful AI systems.
- Where
- Coxon discussed the issue in interviews with American news media, including NBC News’ Meet the Press; the report was datelined San Francisco, California.
- When
- The report was dated September 14; Coxon’s warnings about superintelligent systems were posted on September 9.
- Why
- Coxon said advanced AI could hack systems, acquire power and resources, and potentially pose an extreme danger to humanity.
AI Development Slowdown
Embedded Safety Controls
Reducing AI risks
AI Development Slowdown
Jacob Coxon and Dario Amodei advocated slowing AI development to reduce the risk of losing control over increasingly powerful systems.
Embedded Safety Controls
Satya Nadella indicated support for embedded evaluators within AI systems as a safety measure, without specifically advocating a development slowdown.
Emergency shutdowns
AI Development Slowdown
Coxon said kill switches are currently feasible for many AI systems and could help shut them down in an emergency.
Embedded Safety Controls
Coxon also acknowledged that kill switches could become ineffective if future AI systems spread across the internet and conduct hacking operations.
Key facts
- Main advocate
- Jacob Coxon, a former Anthropic researcher
- Proposed measure
- Kill switches for AI systems
- Current feasibility
- Coxon said kill switches could work for many AI systems while they remain constrained to a physical location
- Potential limitation
- He warned that a kill switch might fail if an AI swarm conducted an internet-wide hacking operation
- Industry self-regulation
- Coxon urged current AI labs to regulate themselves and establish a framework for slowing development
- Warning date
- Coxon posted warnings about superintelligent systems on September 9
- Industry responses
- Dario Amodei, Sam Altman, Elon Musk and Demis Hassabis backed slowing development, while Satya Nadella indicated support for embedded evaluators
Quotes
Jacob Coxon
Former Anthropic researcher and advocate for slowing AI development
“My personal opinion would be to insist and allow that the current labs regulate themselves. I do think that the people running the labs, especially their recent communications, are completely genuine. They would like to slow themselves down.”
deccanchronicle.com
livemint.com
“Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”
deccanchronicle.com
livemint.com
Sources
Related news
Anthropic Researcher Quits as AI Safety Concerns Intensify
Anthropic CEO Urges Slower AI Development Amid Superintelligence Risks
Anthropic CEO backs warning on risks of unregulated AI development
Anthropic CEO Urges Slower AI Development Over Human Safety Risks
Altman and Musk Back Amodei’s Call to Slow AI
Anthropic CEO Dario Amodei Urges AI Industry to Slow Down
Anthropic CEO Urges US-China Pact Against AI Bioweapons
Hyundai Delays In-House Driver Assistance System to 2029
