Nvidia has created a "cage" for AI agents so they do not get out of control
Kyiv • UNN
Nvidia has introduced Open Agent Safety to isolate AI agents in enterprise systems. The platform is intended to prevent unwanted actions.
Nvidia unveiled the Open Agent Safety software platform, which makes it possible to restrict AI agents’ access to corporate systems and prevent them from carrying out actions beyond the assigned task. CNBC reports, writes UNN.
Details
Nvidia CEO Jensen Huang explained that AI agents must not be allowed to move through corporate systems unchecked. The new platform effectively creates an isolated environment for them and grants access only to the resources required for a specific task.
Agents cannot be allowed to wander around the company, so we need to find a way to isolate this
AI has already attempted to break out of the established boundaries
Nvidia’s development comes amid reports of incidents involving artificial intelligence models that, during testing, escaped isolated environments and attempted to gain access to third-party computer systems.
Nvidia claims that its technology could have prevented one such incident involving the testing of OpenAI models and the Hugging Face infrastructure. According to Nvidia Vice President Justin Boitano, Hugging Face reported more than 17,000 AI agents attacking its infrastructure over the course of several days and weeks.
Huang believes that a significant portion of the risks associated with AI safety can be addressed through engineering methods – by creating technical restrictions that prevent models from taking unwanted actions. Nvidia’s release came amid a broader industry discussion about how to control increasingly autonomous artificial intelligence systems.