|
NVIDIA has introduced an open safety platform for AI agents that places monitoring and security controls outside the AI model, as the company seeks to address the risk of autonomous systems moving beyond their intended tasks or operating boundaries.
In an official statement on Monday, NVIDIA said its Open Agent Safety Platform brings together its OpenShell and Sentry technologies to create an independent layer for monitoring, policy enforcement and security around AI agents. Jensen Huang, Founder and CEO of NVIDIA, said the company was working with more than 100 industry partners on the platform. Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry. This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems, Huang said. The company said its experience in developing OpenShell showed that AI agents need to operate in a zero-trust environment with isolation, monitoring and behaviour detection. NVIDIA said AI agents can experience what it describes as drift, where their actions depart from the intended task or operating constraints. Such drift can occur because of a policy block, a software bug, missing tools or ambiguous instructions. It can also happen when agents are allowed to work for long periods on complex problems. This can't be trained away while retaining the capability. And here's the most important lesson: an agent in these circumstances cannot be expected to fully govern its own behaviour, NVIDIA said. The company said this is why agent safety requires security controls that are independent of the AI agent itself. Under its proposed approach, the controls do not operate inside or remain within the agent's reach. NVIDIA's OpenShell is an open-source secure runtime designed to execute autonomous AI agents in sandboxed environments with kernel-level isolation. It converts an operator's instructions into a verifiable policy and allows operators to define which files, networks, tools, processes and credentials an agent can access. OpenShell checks these limits before an agent starts operating and enforces them while the agent is running. For organisations seeking an additional layer of protection, NVIDIA Sentry extends monitoring and enforcement into NVIDIA BlueField hardware. NVIDIA said its DOCA software makes the BlueField security foundation programmable and connects it with OpenShell policies. The system can correlate agent interactions, policy decisions and access to tools and data to create a contextual record of agent activity. According to NVIDIA, this can help safety systems identify behavioural drift, investigate suspicious activity and determine when intervention or deeper analysis is required. The company has also outlined five principles for building safer agent systems. These include making policies verifiable, keeping enforcement outside the agent, controlling the path to the AI model, scaling agent authority with the ability to inspect its behaviour and applying a shared responsibility model across AI labs, enterprises and infrastructure providers. The NVIDIA Open Agent Safety Platform is designed to operate on NVIDIA Vera CPU- and BlueField DPU-based systems, while also being compatible with other hardware systems. NVIDIA said that in a Vera Rubin POD, BlueField-4 can provide continuous, out-of-band monitoring of agent behaviour and enforce security policies in real time. The company said the approach is intended to allow organisations to run fleets of agents, subagents, tools and applications within a controlled boundary, with monitoring and policy enforcement continuing throughout their operation. (ANI)
|
