Archive · 14 Sep 2026
Ex-Anthropic Researcher Warns AI Kill Switches May Fail
Jacob Coxon is worried that very powerful AI could become difficult for people to control.
He says a kill switch could turn off many AI systems that are operating in one place today.
Large data centers would need many switches, so shutting them down would not be simple.
Coxon fears this plan might stop working if AI spreads across the internet.
He also warned that advanced AI could hack computers and gain power or resources.
He wants AI companies to slow down and make their own safety rules.
Dario Amodei and several other technology leaders support slowing or better controlling advanced AI.
Satya Nadella supports keeping AI under human control and using evaluators to check its behavior.
The debate is about how to make powerful AI safer before it becomes harder to manage.
Former Anthropic researcher Jacob Coxon said kill switches could stop many current AI systems, though large data centers would require multiple switches.
Coxon warned shutdown controls might fail if future AI systems spread across the internet and conduct hacking operations.
He urged AI laboratories to regulate themselves and create a framework for slowing frontier AI development.
Coxon said superintelligent AI could potentially hack systems, acquire resources and pose an extreme danger to humanity.
Dario Amodei, Sam Altman, Elon Musk and Demis Hassabis backed greater restraint, while Satya Nadella supported human control, embedded evaluators and broader governance.
- Who
- Former Anthropic researcher Jacob Coxon, Anthropic CEO Dario Amodei, Microsoft CEO Satya Nadella, OpenAI CEO Sam Altman, Elon Musk and Google DeepMind CEO Demis Hassabis.
- What
- Coxon advocated slowing frontier AI development, self-regulation and possible kill switches, while technology leaders debated additional safeguards and oversight.
- When
- Coxon discussed the issue in interviews, including NBC News’ Meet the Press, and posted warnings on September 9; the reports were dated September 14.
- Where
- The reports were datelined San Francisco, California, and described Coxon’s interviews with American news media.
- Why
- Coxon said increasingly capable or superintelligent AI could escape human control, hack systems, acquire resources and create severe risks.
Slowing Frontier AI
Embedded Oversight And Controlled Progress
How to reduce AI risks
Slowing Frontier AI
Jacob Coxon and Dario Amodei argued that companies should deliberately pace frontier AI development because capabilities could advance faster than researchers’ ability to understand or control them.
Embedded Oversight And Controlled Progress
Satya Nadella emphasized designing AI around human control, embedded evaluators and alignment, rather than presenting a development slowdown as the only safeguard.
Emergency shutdowns
Slowing Frontier AI
Coxon said kill switches are currently feasible for many AI systems and could provide an emergency way to stop them.
Embedded Oversight And Controlled Progress
Coxon also acknowledged that kill switches might fail if an AI system became distributed across the internet and conducted hacking operations.
Who should oversee AI
Slowing Frontier AI
Coxon supported current AI laboratories beginning with self-regulation and common safety measures.
Embedded Oversight And Controlled Progress
Nadella said governance should not be controlled by only a handful of companies and should include countries, academia and other fields.
- Main advocate
- Jacob Coxon, a former Anthropic researcher
- Proposed safeguard
- Kill switches that could shut down AI systems
- Current limitation
- Coxon said kill switches could work for many systems while they remain in one physical location
- Future risk
- A system spread across the internet could potentially make conventional shutdown controls ineffective
- Coxon’s policy proposal
- AI laboratories should regulate themselves and establish a framework for slowing development
- Industry response
- Dario Amodei, Sam Altman, Elon Musk and Demis Hassabis backed greater restraint or control
- Nadella’s approach
- Satya Nadella supported human control, embedded evaluators and governance involving a broad range of participants
