3 hrs ago
OpenAI Safety Employee Quits, Says Time for Trial and Error Is Over
David Robinson left OpenAI, a company that develops artificial intelligence.
He says AI companies are moving too quickly and are not being careful enough.
He worries that powerful AI could cause problems that people do not know how to control.
He says companies should learn safety practices from places like nuclear power plants and airports.
He also wants researchers to find better ways to control AI systems that act on their own.
OpenAI says it is improving safety and can pause training or delay releasing models.
The company has also paused training its most advanced models and dropped a planned model release after safety concerns.
Other experts have warned about extreme dangers from AI, while critics say some predictions cannot be tested.
David Robinson resigned from OpenAI and said AI companies are not being careful enough as they develop increasingly capable systems.
He criticized OpenAI’s rapid launch pace and its approach of releasing systems and strengthening safeguards when problems arise.
Robinson called for safety practices informed by industries such as nuclear power and aviation, and research into controlling autonomous AI systems.
OpenAI said it is strengthening safety and security, and will pause training or hold back models when needed; it also paused training its most advanced models and scrapped a planned next-generation model release after internal safety concerns.
Other warnings included Geoffrey Irving’s estimate of a 50% chance of human death from smarter-than-human AI; critics said such predictions cannot be verified or falsified.
- Who
- David Robinson, a former OpenAI safety employee, criticized the company’s approach; OpenAI and other AI researchers also feature in the debate.
- What
- Robinson resigned and called for stronger safety practices and research as AI systems become more capable.
- Where
- Robinson’s essay was published in The Atlantic; the resignation concerned OpenAI.
- When
- His essay was published on Saturday; the articles also describe OpenAI decisions in recent weeks and this week.
- Why
- Robinson said rapid development and an overly optimistic approach to solving problems later could allow safety failures to grow.
Calls for stronger safeguards
Company response and skeptical views
Whether AI development is careful enough
Calls for stronger safeguards
Robinson said AI firms are not being nearly careful enough and argued that rapid launches and optimism about fixing problems later could worsen safety failures.
Company response and skeptical views
OpenAI said it is strengthening safety and security, ensuring models do not exceed what it can safely manage, and pausing training or withholding models when needed.
How to manage AI risks
Calls for stronger safeguards
Robinson called for safety expertise from nuclear power and aviation, along with research to ensure autonomous AI systems can be controlled.
Company response and skeptical views
The articles report that OpenAI has paused training and scrapped a planned model release after safety concerns, but do not say whether it adopted Robinson’s specific proposals.
Predictions of catastrophic harm
Calls for stronger safeguards
Geoffrey Irving estimated about a 50% chance that humanity could die because of smarter-than-human AI; Anthropic warned of more than a 10% chance of human extinction within a decade.
Company response and skeptical views
Critics said such predictions are unscientific because they cannot be verified or falsified.
Key facts
- Resigning employee
- David Robinson, who said he spent 3-1/2 years at OpenAI and oversaw safety reports for 12 frontier-model launches.
- Essay
- “I Quit OpenAI Because Its Culture Is Broken,” published in The Atlantic.
- Robinson’s criticism
- He said AI companies are not being nearly careful enough and that the time for trial and error is over.
- Safety proposals
- Robinson called for expertise from fields such as nuclear power and aviation, and research into reining in autonomous systems.
- OpenAI actions
- The company said it had paused training its most advanced models and scrapped a planned next-generation model release after researchers raised safety concerns.
- OpenAI response
- A spokesperson said the company was strengthening safety and security and would pause training or hold back models when needed.
- Other risk estimates
- Geoffrey Irving estimated about a 50% chance that humanity could die because of smarter-than-human AI; Anthropic separately warned of more than a 10% chance of AI wiping out humanity within a decade.
Quotes
David Robinson
Former OpenAI safety employee who resigned and wrote about the company’s AI safety culture
“I agree with other recently departed staff that the companies building this technology aren’t being nearly careful enough. But I believe that we need to look deeper than specific rules or new laws.”
firstpost.com
“As the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed.”
CNBC TV 18
OpenAI spokesperson
Spokesperson for OpenAI.
“We’re making sure our models don’t become more capable than we can safely manage and secure, and we pause training or hold back models when we need to slow down.”
CNBC TV 18
firstpost.com
“Recent warnings about the potential destructive power of AI are understating the severity of the situation.”
firstpost.com
Sources
Related news
Anthropic Researcher Quits as AI Safety Concerns Intensify
OpenAI Considers Slowing Cutting-Edge AI Amid Safety Concerns
OpenAI Parts Ways With Three Researchers Over Confidential Data
AI Safety Researchers Leave Labs Over Humanity Risks
OpenAI Pauses AI Training After Rogue Agent Incidents
OpenAI Shelves GPT-6.1 Astra After Safety Tests Fall Short
