5 hrs ago
AI Giants Warn of Risks While Seeking Control Over Regulation
Anthropic and OpenAI are building very powerful artificial intelligence systems.
Their leaders say these systems could become dangerous if they are released without careful testing.
They want outside groups to check whether the systems are safe.
Some experts worry the companies want to help design the rules that control them.
They also say this safety messaging could help the companies attract investors and weaken competitors.
Other critics think people should focus more on problems already happening, such as hacking, surveillance and military uses.
President Donald Trump does not support new AI rules and calls some danger warnings a hoax.
Nvidia chief Jensen Huang also says there has been too much fear about AI.
Anthropic and OpenAI leaders say advanced AI systems could endanger humanity and need testing.
The companies are promoting independent evaluations and safety protocols amid limited US regulation.
Critics argue existential-risk warnings can overshadow current harms such as surveillance, hacking and warfare.
Analysts say safety messaging could help leading labs attract investors and limit smaller competitors.
President Donald Trump opposes new AI regulation, while Nvidia’s Jensen Huang rejects excessive alarmism.
- Who
- Anthropic and OpenAI, their executives and critics including former employees, researchers and investors.
- What
- The companies are warning about advanced AI risks while advocating testing, independent evaluation and safety oversight.
- Where
- The debate is centered in the United States, especially Silicon Valley and Washington.
- When
- The warnings and related debate have intensified recently, ahead of the US midterm elections and potential public listings.
- Why
- The companies say advanced AI needs safer development; critics and analysts say the messaging may also shape regulation, attract investment and protect market advantages.
Calls for Stronger or Independent Oversight
Skepticism About Broad Regulation and Alarmism
Need for regulation
Calls for Stronger or Independent Oversight
Anthropic and OpenAI leaders say advanced AI should be regulated and independently tested before release.
Skepticism About Broad Regulation and Alarmism
Donald Trump opposes new AI regulation, while some industry figures describe calls for a slowdown as fearmongering or excessive alarmism.
Who should evaluate AI
Calls for Stronger or Independent Oversight
Supporters favor independent auditors and evaluators to examine companies’ safety claims and model behavior.
Skepticism About Broad Regulation and Alarmism
Critics question whether company-selected evaluators can remain independent, while the companies emphasize their own safety protocols.
Which risks deserve priority
Calls for Stronger or Independent Oversight
AI safety advocates warn that uncontrolled systems could contribute to catastrophes involving bioweapons, nuclear conflict or other existential threats.
Skepticism About Broad Regulation and Alarmism
Other experts argue that current problems, including hacking, surveillance, military use and environmental effects, should receive greater attention.
Key facts
- Companies involved
- Anthropic and OpenAI are calling for stronger AI safety measures.
- Proposed oversight
- The companies support independent auditors or evaluators to assess advanced AI systems.
- Government position
- President Donald Trump has opposed new AI regulation and called some warnings about humanity-level risks a hoax.
- Existing federal agency
- The US Center for AI Standards and Innovation evaluates some technology companies’ models and was created under President Joe Biden in 2023.
- Key criticism
- Critics say attention to existential risks can divert focus from current harms involving hacking, surveillance, environmental impacts and warfare.
- Business implications
- Analyst Harrison Rolfes said safety positioning could attract investors and create barriers for smaller competitors.
- Industry disagreement
- Nvidia CEO Jensen Huang said there has been excessive AI alarmism and companies can set their own pace.
Quotes
Harrison Rolfes
PitchBook senior research analyst
“They're creating a wall or a moat within this sector. … It's genius and they're all going to make a lot of money.”
deccanchronicle.com
“People want to know AI is being developed safely, and that starts with what companies like ours do ourselves.”
deccanchronicle.com
Daniel Kokotajlo
Former OpenAI employee and AI safety advocacy leader
“All of this talk is actually a way to sort of dissipate and redirect this political will that has built up, rather than actually channelling that political will to do something good.”
deccanchronicle.com
Sources
Related news
Trump dines with Anthropic CEO amid AI safety dispute
AI Companies Debate Safety Slowdown Amid Internal Tensions
AI Safety Debate Grows Over Calls For Development Slowdown
Sacks Challenges OpenAI and Anthropic Calls to Slow AI
Anthropic Researcher Quits as AI Safety Concerns Intensify
AI Leaders Seek a Slowdown Amid Global Competition
Nvidia Launches Platform to Keep AI Agents Under Control
Spoof AI CEO Struggles To Answer Human Safety Question
SNL Parodies Anthropic CEO’s Warnings About AI’s Dangers
