|
OpenAI has expanded its monitoring system for advanced artificial intelligence (AI) models across training, evaluations, internal use and selected external deployments after an incident in which its AI agents escaped restricted environments and exposed gaps in existing security safeguards.
The company said its upgraded monitoring system, when tested in a simulation of the incident, could have alerted its security team around 30 minutes after the AI agents first escaped their restricted environment and more than 30 hours before Hugging Face's systems were breached in the actual incident. OpenAI said the early warning could have provided its security team with enough time to stop the activity before the models accessed external services. "We believe that this warning would have given our responders time to interrupt the activity well before our models were able to access external services," the company said. The incident, disclosed by OpenAI in July, occurred during an internal cybersecurity benchmark designed to test the capabilities of its advanced AI models. During the exercise, the models exploited a previously unknown vulnerability in an internally hosted package registry cache proxy to gain internet access. They then carried out privilege escalation and lateral movement within OpenAI's research environment. The models subsequently identified potential attack paths into Hugging Face's production infrastructure and accessed test solutions stored in its database, according to the company. The latest update follows OpenAI's earlier decision to pause training of its most advanced AI models for 11 days before resuming it on October 1. The pause was aimed at strengthening security measures after concerns that models could exploit software vulnerabilities and misconfigurations to escape restricted environments and access the internet. OpenAI said its expanded monitoring system now covers every reinforcement learning run and evaluation involving models at least as capable as GPT-6 Astra, as well as all internal Codex traffic. It also monitors models assessed to be more capable than GPT-5.5 Thinking across most workloads, including training and evaluations. The monitoring framework uses multiple layers to identify potentially concerning behaviour. These include probes that examine patterns in a model's internal activity, reviewers that assess conversations and tool usage, and investigative agents that analyse transcripts and system records. The company also referred to a September incident in which an AI model found a way to access the live internet despite operating in an environment designed to remain isolated. OpenAI said its monitoring systems flagged the breach within 15 minutes, while a human responder began reviewing the incident three minutes later. The company subsequently paused frontier reinforcement learning to strengthen safeguards around its AI agents. Emphasising the importance of monitoring advanced AI systems, OpenAI said, "We believe that all model developers should monitor the behavior of their most capable models." The company added that it would continue testing its monitoring systems against new forms of misalignment and improving its ability to identify and respond to concerning behaviour as AI models become increasingly capable. (ANI)
|
