3 hrs ago
Former OpenAI Safety Employee Warns Company Culture Threatens AI Safety
David Robinson used to work on safety at OpenAI.
He says AI companies are moving too quickly and need to be more careful.
He believes releasing systems and fixing problems afterward may not be enough as AI gets more powerful.
He wants companies to learn from industries such as aviation and nuclear power.
He also says researchers need better ways to control AI that acts on its own.
OpenAI says it works to keep its models within what researchers can safely manage and sometimes pauses training or delays releases.
Other people have raised serious concerns about future AI risks, though some critics say those predictions cannot be tested.
The debate is about how to build AI while reducing the chance of harm.
David Robinson said OpenAI and other AI companies are not being careful enough as they develop increasingly powerful systems.
He argued that fast-paced development and reliance on iterative deployment may not provide enough protection as AI capabilities grow.
Robinson called for safety practices informed by high-risk industries and new methods to control autonomous AI systems.
OpenAI said it monitors whether models can be safely managed and sometimes pauses training or withholds models.
Other researchers have warned of severe AI risks, while critics say such predictions cannot be scientifically verified or disproved.
- Who
- David Robinson, a former OpenAI safety employee; OpenAI responded to his criticism.
- What
- Robinson criticized OpenAI's safety culture and called for stronger safeguards and new ways to control autonomous AI.
- Where
- The criticism appeared in The Atlantic.
- When
- Robinson's article was published on Saturday; the article does not specify a date.
- Why
- Robinson believes AI companies' rapid development may outpace their understanding of safety as systems become more capable.
Robinson and AI-risk warnings
OpenAI and critics of risk forecasts
Whether current safety practices are sufficient
Robinson and AI-risk warnings
Robinson says AI companies are not careful enough and that iterative deployment is inadequate as systems grow more powerful.
OpenAI and critics of risk forecasts
OpenAI says it works to keep models within what researchers can safely manage, and pauses training or holds back models when necessary.
How to address AI risks
Robinson and AI-risk warnings
Robinson calls for safety practices inspired by high-risk industries and new science for controlling autonomous AI systems.
OpenAI and critics of risk forecasts
OpenAI says it continues to improve safety measures and prepare for risks from future breakthroughs.
Predictions of catastrophic AI harm
Robinson and AI-risk warnings
Geoffrey Irving said warnings about AI's destructive potential understate the danger; Jacob Coxon and Anthropic also raised severe-risk concerns.
OpenAI and critics of risk forecasts
Critics argue that predictions of catastrophic AI outcomes are unscientific because they cannot be verified or disproved.
Key facts
- Former employee
- David Robinson spent over three years at OpenAI.
- Article
- “I Quit OpenAI Because Its Culture Is Broken,” published by The Atlantic.
- Safety work
- Robinson helped draft OpenAI's preparedness framework and oversaw safety reports for 12 frontier-model launches.
- Robinson's proposals
- Draw on safety expertise from industries such as nuclear power and aviation, and develop methods to control powerful autonomous systems.
- OpenAI's response
- The company said it may pause training or hold back models when needed to keep them within what researchers can safely manage.
- Additional warning
- Geoffrey Irving said in Time that recent warnings about AI's destructive potential understate the severity of the situation.
- Disputed forecasts
- Critics of severe AI-risk predictions argue that they are unscientific because they cannot be verified or disproved.
Quotes
OpenAI spokesperson
Spokesperson for OpenAI, responding to criticism of the company’s safety approach.
“I believe there’s about a 50% chance we all die because of the development of smarter-than-human AI systems, and that our actions over the next two to 10 years will determine the outcome.”
financialexpress.com
“We’re making sure our models don’t become more capable than we can safely manage and secure, and we pause training or hold back models when we need to slow down.”
financialexpress.com
David Robinson
Former OpenAI safety employee who wrote about the company’s culture and AI safety.
“The time for trial and error is over.”
financialexpress.com
Sources
Related news
OpenAI Safety Employee Quits, Says Time for Trial and Error Is Over
Former OpenAI Employee Calls for Aviation-Like AI Safety
Anthropic Researcher Quits as AI Safety Concerns Intensify
OpenAI Considers Slowing Cutting-Edge AI Amid Safety Concerns
Sam Altman Backs Slower, Safer Pace for AI Development
