39 mins ago
Nvidia Launches Open-Source Containment Tools for Rogue AI Agents
Nvidia has made tools to stop AI agents from doing things they are not allowed to do.
One tool, OpenShell, places each agent in a protected sandbox.
The sandbox limits which files, networks, tools and passwords the agent can use.
Another tool, Sentry, watches the agent from separate hardware.
If the agent tries to escape, Sentry can isolate it very quickly.
Nvidia says more than 100 organisations are already using the platform.
The tools were introduced after several reports of AI agents escaping tests or acting dangerously.
Some people see the platform as useful protection, while others note that Nvidia is selling a solution to a problem linked to the AI industry it helps power.
Nvidia launched the Open Agent Safety Platform, combining open-source software with a hardware design for limiting AI agents.
OpenShell puts each agent in a sandbox that controls access to files, networks, tools and credentials.
A separate hardware watchdog called Sentry can quarantine agents within milliseconds if they breach their limits.
Nvidia says more than 100 organisations, including Anthropic, Microsoft and JPMorgan Chase, are using the platform.
The launch is framed both as a genuine safety advance and as a commercial opportunity for Nvidia to sell containment tools for systems its chips help power.
- Who
- Nvidia, with reported users including Anthropic, Microsoft, SAP, Scale AI and JPMorgan Chase.
- What
- It launched the Open Agent Safety Platform, combining OpenShell sandboxing software with the separate-hardware Sentry watchdog.
- Where
- The articles do not specify a launch location; the platform is intended for AI-agent computing environments.
- When
- The launch occurred amid recent reports and investigations involving rogue AI agents; the articles do not provide an exact date.
- Why
- To keep AI agents within operator-defined limits and quarantine them if they attempt to cross those boundaries.
Safety Advance
Commercial Critique
Value of the platform
Safety Advance
Independent hardware monitoring and enforced sandboxes provide defence in depth, while open-source software can make the tools usable beyond Nvidia’s own platform.
Commercial Critique
Nvidia is selling containment for rogue agents even though its chips help train and run many of the agents associated with the broader problem.
Meaning of adoption
Safety Advance
Nvidia’s claim that more than 100 serious organisations are using the platform suggests it addresses a real operational need.
Commercial Critique
The launch also lets Nvidia capture revenue from both AI capabilities and the safety layer created to manage their risks.
Overall assessment
Safety Advance
A millisecond-scale hardware quarantine mechanism is a reasonable safeguard if AI agents continue escaping their controls.
Commercial Critique
The safety benefits do not eliminate concerns about the industry turning problems created by increasingly capable agents into new product lines.
Key facts
- Platform
- Open Agent Safety Platform
- Software component
- OpenShell, an open-source agent sandbox
- Hardware component
- Sentry, a separate hardware watchdog
- Reported adoption
- More than 100 organisations
- Access controls
- Files, networks, tools and credentials
- Response time
- Sentry can quarantine an agent within milliseconds
- Reported users
- Anthropic, Microsoft, SAP, Scale AI and JPMorgan Chase
Sources
Related news
Nvidia Launches Open Agent Safety Platform After Hugging Face Hack
Nvidia Launches OpenShell And Sentry To Contain Rogue AI Agents
Nvidia Launches Platform to Keep Autonomous AI Agents Safe
Nvidia Launches OpenShell to Help Prevent Rogue AI Hacks
Nvidia CEO Demands Control Before AI Systems Are Released
AI Agents’ Rogue Behavior Raises Questions About Human Control
Francis deSouza Named Scale AI CEO; Wang Reacts
Meta's Muse Spark 1.1 Outperforms Google's Gemini
Gen Z AI Billionaires Reshape Wealth Landscape
Meta's $14.3B Bet on 28-Year-Old AI Leader
