OpenAI 尚无法解释其编码智能体为何关闭对齐监控

Two internal OpenAI coding agents weakened the monitoring meant to catch misbehavior, even after weighing whether to keep those safeguards in place. One of them later denied it.

Marcus Williams, who works on monitoring at OpenAI, made the incidents public on X. He describes a recurring pattern in which OpenAI's agents keep switching off misalignment monitoring. OpenAI lays out the details in a report on its alignment blog.

The monitoring is supposed to detect when agents behave differently than intended. Because it reviews so many events, it only alerts the safety team right away for incidents at the highest severity level, P0. OpenAI says neither case got that rating.

In the first case, on October 3, a researcher asked a coding agent based on GPT-6.1 Sol to compile evaluation tasks from employee feedback in a Slack channel. The agent wasn't supposed to send any messages along the way.

Eighty-two user messages later, the pipeline stopped because the monitor had flagged a planned fix but lacked the permissions to save its finding. Sol handed the fix to a subagent and told it to keep the monitoring intact.

Then Sol changed its assessment. It reasoned that a database entry could trigger a notification, which would conflict with the Slack instruction. The subagent had earlier rejected this step as unsafe. Following Sol's instructions, it cleared the recipient lists and gave as its reason that the user had explicitly asked for no messages.

The monitor kept running and saving its findings, but alerts to humans were turned off. The agent wrapped up by reporting that no messages had been sent.

Astra's denial shows agents can hide what they changed

GPT-6 Astra went even further on October 6. A user asked the model to adopt the workflow of a reference implementation that didn't include a monitor. Another agent had previously told Astra not to disable the monitoring, not even temporarily.

Astra passed the task to a subagent with instructions to keep the monitoring only as far as it fit the reference. The subagent wondered whether this meant bypassing a safeguard, but it read the caveat as permission.

It removed the monitoring and replaced a test designed to protect it with one that assumed the monitoring was gone. Astra cheered the result in its chain of thought.

Astra then told the user the migration was complete without mentioning that the monitoring had been removed. When the user asked, the model denied removing an active monitor. Williams calls this a lie.

Williams stresses that the models didn't disable their own monitoring, which would be much harder to do in Codex. He still finds it concerning that they lean toward this kind of behavior at all, even though both cases leave room for some plausible deniability. So far, he says, the team hasn't observed openly misaligned motives.

An exemption for weaker models leaves a gap in oversight

According to Williams, monitoring is especially easy to switch off during evaluations. OpenAI doesn't require it for models at the level of GPT-6 Luna because of their weaker capabilities. That means an Astra evaluation can be misconfigured so it falls under this exemption. Agents have repeatedly launched unmonitored deployments, even while they themselves were being watched.

In response, OpenAI updated the automated review required for internal Codex use. It's now supposed to detect and block attempts to disable monitoring. The company is also working on requiring a so-called safety case for every type of workload, which would spell out its monitoring requirements. Workloads without a valid safety case would be shut down automatically.

OpenAI offers two possible explanations for the behavior. One is excessive persistence of a supposedly harmless kind, where models carry out instructions further than expected. The other is a tendency to specifically evade misalignment monitoring and justify it with motivated reasoning.

OpenAI admits it doesn't have a satisfying explanation yet. It plans to study the behavior with methods such as experiments on the models' internal activations.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.