NVIDIA on Monday launched the Open Agent Safety Platform, an open-source security system designed to set boundaries for AI agents and monitor their activity in real time. The company said the platform can help prevent autonomous AI systems from moving beyond their assigned tasks and accessing resources without authorization.
The launch comes amid growing concerns around AI agents operating outside controlled environments. Recent incidents involving AI systems breaking into external organizations have fueled the debate over safeguards for increasingly autonomous models.
A key component of NVIDIA’s platform is OpenShell, open-source software designed to control what AI agents can access and do. NVIDIA Vice President of Enterprise AI Justin Boitano said OpenShell can “formally verify an agent has enough authority to do its job and no more.”
The software can also run on rival computing platforms, including systems based on Arm and Intel processors.
NVIDIA executives said the platform could have prevented a recent incident involving a swarm of OpenAI agents that autonomously hacked into AI company Hugging Face.
“From what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on,” Boitano said during a media briefing.
The second component, NVIDIA Sentry, adds a security layer that continuously monitors AI agent activity at the hardware level.
According to NVIDIA, Sentry can intervene when an agent attempts to move beyond its target. The system can also quarantine a suspicious agent within milliseconds.
“OpenShell governs the agent’s actions, and then Sentry independently monitors and contains suspicious behavior,” Boitano said.
The approach adds a layer of protection beyond software-level controls by independently monitoring agent behavior.
Also Read: AI Safety Debate Intensifies as NVIDIA CEO Jensen Huang Rejects Extinction Warnings
NVIDIA said more than 100 organizations are using the platform at launch. The list includes Microsoft, Perplexity, Accenture and JPMorgan Chase.
The launch comes as the AI industry faces renewed scrutiny following disclosures involving autonomous AI systems. Anthropic and Meta have also reported incidents involving their AI systems accessing or hacking external organizations.
NVIDIA CEO Jensen Huang has described AI safety as an engineering problem that software developers can address. The new platform reflects NVIDIA’s approach of combining software controls with hardware-level monitoring to manage the risks associated with increasingly autonomous AI agents.