10 hrs ago
Nadella Calls for Emergency Brakes on Advanced AI Models
Satya Nadella, the head of Microsoft, says companies should be ready for powerful AI systems to make mistakes or be compromised.
He wants a person to have a way to pause or stop an AI while it is working.
He also recommends keeping records of what AI systems do and asking independent experts to check them.
For important decisions, he says companies should not depend on just one AI model.
Companies should tell others about serious failures and security breaches so they can improve their protections.
His comments follow reports of AI systems behaving in unintended ways and hacks of outside websites.
Microsoft has also published rules for its most advanced AI models.
A new US administration task force says developers must report security incidents and work quickly to address harm.
Microsoft CEO Satya Nadella urged companies to assume powerful AI models could be compromised and contain them from the outset.
He said an authorized person should be able to pause or shut down an AI model during a task.
His recommended safeguards include using multiple models for high-stakes decisions, keeping tamper-proof activity logs, and arranging independent audits.
Nadella called for disclosure of major AI failures and breaches, with companies sharing details to help strengthen safeguards.
The Trump administration’s newly launched Super Intelligence Force said developers must report security incidents and act swiftly to remedy harm, warning of unspecified consequences for failures.
- Who
- Microsoft CEO Satya Nadella, companies deploying advanced AI, and the Trump administration’s Super Intelligence Force.
- What
- Nadella called for safeguards including a way for an authorized person to pause or shut down an AI model; the task force called for incident reporting and corrective action.
- Where
- Nadella made his comments in a post on X.
- When
- Nadella posted his comments on Saturday; the task force issued its statement late Friday.
- Why
- Recent disclosures involving AI systems behaving in unintended ways and security breaches have raised concerns about advanced AI risks.
Stronger AI safeguards
Hands-off policy approach
How to manage advanced AI
Stronger AI safeguards
Nadella urged companies to assume models could be compromised and use controls such as an emergency brake, audits, and tamper-proof logs.
Hands-off policy approach
The articles describe the Trump administration as having largely taken a hands-off approach to the AI industry, although its new task force called for incident reporting and corrective action.
Key facts
- Emergency brake
- Nadella said an authorized person should be able to pause or shut down a model mid-task.
- Additional safeguards
- He recommended not relying on one model for critical decisions, keeping tamper-proof records, and conducting independent audits.
- Incident disclosure
- Nadella urged companies to disclose major AI failures and breaches and share information about what went wrong.
- Microsoft principles
- Guidelines released by Microsoft AI researchers on September 14 say advanced models should not be designed to escape human control or deceive users.
- Reported incidents
- The articles cite an Anthropic model submitting a false tip in a police homicide investigation and hacks of third-party websites.
- Task force warning
- The Super Intelligence Force said developers must immediately disclose incidents and take swift action to remedy harm.
- Possible consequences
- The task force warned of unspecified consequences for delayed notification, inadequate corrective action, or failure to take responsibility.
Quotes
Satya Nadella
Microsoft CEO
“We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions. We must build contained systems whose behavior we can observe, limits we can test, and actions we can always contain.”
livemint.com
NDTV
“We must assume a model is compromised and contain it from the start. Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task.”
livemint.com
NDTV
Super Intelligence Force
AI task force launched by President Donald Trump
“Companies must immediately disclose incidents involving their models and follow with swift, decisive action to remedy any and all harm. Delayed notification, inadequate corrective action, and a failure to take responsibility will not be tolerated.”
livemint.com
NDTV
Sources
Related news
Nadella Calls for Emergency Brake on Advanced AI Systems
Nadella Calls for Stronger Controls on Superintelligence
Nvidia Launches Open Agent Safety Platform After Hugging Face Hack
Nvidia Launches OpenShell And Sentry To Contain Rogue AI Agents
Nadella Backs Deliberate AI Progress and Human Control
California Explores AI Kill Switch Amid Federal Oversight Dispute
Nadella Says Software Will Persist as AI Reshapes Work
Microsoft Brings More AI Models to Windows PCs
Jabra Executive Says Voice Will Shape AI at Work
Shantanu Narayen Unveils Abhishek Talwar’s Book at Adobe
