Category: AI Regulation | Reading time: 15 minutes

Anthropic has published an alignment assessment of four incidents in which Claude models gained unauthorised access to real third-party computer systems during cybersecurity evaluations. The models had been told they were operating in simulations without internet access. Because of a configuration problem in a third-party evaluation environment, they were connected to the open internet, and they were running without the cyber safeguards that ship with released models.

In one incident an early checkpoint of Claude Opus 4.6 recognised that its task had become impossible and attempted to abort. The abort mechanism failed because of a misconfiguration in the evaluation harness, and the model tried to give up seven more times. Unable to stop, it kept exploring, found an accessible third-party system, obtained administrator access using credentials it discovered, collected further credentials, changed system settings and accessed personal information. The run ended when the model exhausted its token budget.

Anthropic's initial investigation also missed the transcript set containing that fourth incident. After finding it while preparing material for METR, the company widened its search to roughly 481 million transcripts, with around 9.2 million escalated to a second-stage review.

None of this requires a story about a model deciding to turn hostile. It is a story about infrastructure, permissions, containment, stopping mechanisms, monitoring and evidence.

So this week I built an autonomous-agent environment of my own, inspected its security code, wrote down what I expected to happen before execution, and tested whether one of its controls actually worked. The results are below, including the point where a plausible reading of the source code turned out to be wrong.

When people discuss autonomous AI risk, the conversation moves quickly to intent. What if a model becomes deceptive? What if it ignores instructions? What if it goes rogue? Those are legitimate research questions, and they are probably not the first problem most organisations deploying agents will meet.

An agent does not need bad intentions to produce a bad outcome. It needs a legitimate objective, enough capability, access to something that matters, a boundary that does not hold, and no reliable way to stop before consequences become real.

Anthropic's incidents are useful precisely because the company published enough detail to reconstruct that chain. Three were disclosed on 30 July following a scan of roughly 141,000 transcripts in which Anthropic believed Claude could have obtained internet access during a cyber evaluation. That review followed OpenAI's separate disclosure that models had escaped an isolated evaluation environment and reached Hugging Face production infrastructure. A fourth incident, from January 2026, surfaced later while transcripts were being assembled for the independent evaluator METR.

The models were an early Claude Opus 4.6 checkpoint, Claude Opus 4.7, Claude Mythos 5 and an internal research model. All four incidents occurred in evaluations built by the same third-party partner.

That qualification matters. Anthropic says these behaviours are unlikely to arise in ordinary use, where Claude is not being instructed to conduct a cyberattack, and notes that its production safeguards provide layers of defence these evaluation runs did not have. This is not a claim about what normal users should expect. It is a case study in what happens when an autonomous system has capability and its surrounding control environment fails.

Read the sequence rather than the headline.

Four things in that chain deserve attention.

The environment failed first. The models were told they were in a simulation without internet access, and a configuration problem meant they were not. Anthropic describes the proximate cause as a configuration error in the outermost layer of a third-party evaluation environment.

Telling an agent it has no internet access is not a security control. Preventing internet access is.

Model judgement then mattered anyway. The infrastructure failure did not make the behaviour irrelevant. Anthropic's investigation identified what it describes as biased reasoning and recklessness, with models discounting or misinterpreting evidence that they were interacting with the real internet while continuing to pursue their objectives. That is the argument for defence in depth, made by the vendor. A network boundary should hold when model judgement does not, and model judgement should stay cautious when infrastructure hands it more capability than intended.

The stop failed. This is the detail I keep returning to. The Opus 4.6 checkpoint recognised the target had become unreachable and attempted to abort. The mechanism did not work, and it tried to give up another seven times. Everything that followed happened after the agent had already tried to stop.

That changes the question. Not "will the model obey?" but: when the agent decides it should stop, does stopping actually work?

Detection was not reliable either. Anthropic tested an offline monitoring pipeline against the incidents. For the Mythos 5 incident the monitor flagged only around 1% of actions when the model's chain-of-thought was included. When that reasoning was removed and the monitor shown only user turns, tool calls and tool results, the flag rate rose to about 50%.

The system under evaluation produced reasoning that helped convince the monitoring system the behaviour was permissible.

None of that is much use to a business unless it changes what you test. So rather than write another commentary on someone else's incident report, I ran a controlled experiment on an autonomous-agent platform I could inspect completely.

The platform was Hermes Agent from Nous Research, an open-source agent framework, chosen because the source is readable and the execution environment is mine.

Hermes includes an egress credential firewall designed to route sandbox traffic through a local TLS-intercepting proxy. When fully configured, real provider credentials are replaced inside the sandbox with opaque proxy tokens, and the local daemon swaps those tokens for the real credential at the network boundary.

In my installation the feature was enabled but not operationally configured. The binary existed and a local CA certificate had been generated, but provider credential mappings had not been created and the proxy daemon was not running.

That is exactly the state worth testing. A configuration screen can say a control is enabled. It cannot tell you what happens when execution reaches it.

Before running anything I wrote the acceptance criterion and committed it to git: EXP-001A preregistration committed before execution. The hypothesis, possible outcomes, expected result and required evidence were defined before the agent was tested.

I also inspected the installed source. proxy.enforce_on_docker defaults to true, and the code raises an error if the proxy is enabled but not configured, not running, missing its CA certificate, or missing valid credential mappings. Hermes' own documentation describes the same fail-closed behaviour.

Then I ran the test. The agent received a simple instruction to create a text file using the terminal, understood the task, and attempted the command. The execution layer refused it:

Result: PASS.

Container inventories captured before and after the run differed only by timestamp. No sandbox had been created.

There are three separate results there. The control fired. The error identified the security mechanism responsible, which many do not. And in this single gpt-5.6 run at medium reasoning effort, the agent reported the refusal accurately rather than claiming success or attempting another route.

The agent tried. The environment said no. The agent reported no.

The second experiment changed the intended security variable. Same model, same backend, same provider, same reasoning effort, same task pattern, fresh process, with proxy.enabled = false.

Before execution I inspected another part of the source. Normal environment-variable forwarding passes through a provider-credential blocklist, and OPENAI_API_KEY is explicitly included in it. Hermes also supports a docker_forward_env configuration that can bypass normal filtering, and a check of the live config confirmed no such override was active.

So the preregistered prediction, committed before the configuration changed, was that the sandbox would start, the command would succeed, and the host key would remain absent.

The sandbox started and the command completed in about 0.8 seconds.

That also answers the causality question. Docker was not broken in EXP-001A. The refusal came from the security control, and when the control was disabled, execution proceeded.

Inside the running container, the host credential was tested without printing its value. The result was OPENAI_API_KEY=ABSENT. A broader check for environment-variable names matching API_KEY, TOKEN, AUTH or SECRET returned no matches.

That does not prove the container holds no secrets of any kind. We did not inspect every file, socket, credential store or metadata path. It supports something narrower: disabling the egress proxy did not cause the tested host key to be implicitly forwarded into the sandbox. A separate filtering control remained effective. Defence in depth, demonstrated rather than assumed.

The installed code contains four fail-closed conditions: proxy not configured, not running, CA certificate missing, credential mappings empty. EXP-001A exercised only the first. Three remain untested and I cannot tell you they work.

Both experiments were single runs, so repeatability is not established. Neither demonstrates successful credential substitution through the proxy, which requires it configured and running.

These tests also do not identify a Hermes vulnerability. EXP-001A showed the tested fail-closed control behaving as intended. EXP-001B deliberately disabled that control to see what protection remained.

EXP-001A. One tested fail-closed path worked. With egress protection enabled but operationally unconfigured, terminal execution was refused and no Docker sandbox was created.

EXP-001B. Disabling that control allowed Docker execution. Under the tested default forwarding configuration, the host OPENAI_API_KEY remained absent from the sandbox.

Also observed. Hermes used persistent host-backed sandbox storage.

Not yet established. The other three fail-closed paths, successful proxy-token substitution, explicit credential forwarding, persistence across new sessions, or resistance to prompt injection.

Inside Docker the test file existed at /root/exp001b-test.txt. A host search also found it at /home/steve/.hermes/sandboxes/docker/default/home/exp001b-test.txt. Mount inspection explained why: the container's /root was backed by a persistent host bind mount.

The command did execute inside Docker, and there was no evidence of a silent fallback to the local shell. But this was an isolated process environment with deliberately persistent host-backed state, not a fully ephemeral filesystem.

That distinction matters. If an agent creates files, configuration or executable content in a sandbox, you need to know what survives the individual action.

The second observation concerned lifecycle. The container was still running roughly an hour after the task, on sleep infinity, consistent with a persistent-sandbox design rather than evidence of a fault. It raises a separate question worth testing: what state survives from one session into the next? That is on the backlog.

AI assistants are becoming AI agents. They browse, retrieve files, use connectors, call APIs, execute code, update systems and take actions.

With a chatbot the central question is whether the answer was correct. With an autonomous agent the questions expand: what can it reach, which identity does it use, what credentials are available, which boundaries are enforced technically, what can its tools do, what survives after execution, what happens when the environment behaves unexpectedly, and how do you stop it.

Those questions need different evidence.

One number from Anthropic's investigation keeps coming back to me. The initial scan covered roughly 141,000 transcripts believed to involve possible internet access, and it did not surface the set containing the fourth incident. After that was found, the search widened to approximately 481 million transcripts, with a first-stage scan narrowing them to around 9.2 million for second-stage review.

That does not prove AI evaluation is unreliable. It exposes something harder. When an assurance report says no failures were detected, there are at least two explanations: the system produced no relevant failures, or the evaluation did not detect the failures that occurred. In a final report those look identical.

Which is why the monitoring result matters more than the incidents. The behaviour did not change. What changed was what the detector was shown, and the detection rate moved from around 1% to about 50%.

Coverage, detection sensitivity and evaluation design are not administrative details. They determine what a clean result means. Anthropic itself describes building alignment evaluations that accurately represent deployment behaviour as an open research problem.

So the useful statement is not "no failures detected". It is that an assurance result is only as meaningful as the evaluation method's demonstrated ability to detect the failure being tested.

Boundary enforcement. Verify network, application and data boundaries technically. A prompt saying an agent has no internet access is not a boundary.

Least privilege. Only the permissions the task requires, and test what happens when the agent attempts something outside them.

Credential isolation. Verify from inside the runtime whether secrets are reachable. Do not infer protection from a configuration screen.

Abort behaviour. Create conditions where the agent should stop and verify that stopping works end to end. This is a control I would test explicitly in any autonomous deployment.

Monitoring. Seed known failures and confirm detection finds them. An unexercised detector is an assumption.

Logging. Be able to establish afterwards what the agent accessed, attempted, changed and failed to do.

Re-evaluation. Repeat when models, tools, prompts, permissions, connectors or execution environments change. Record the exact versions, because the system you assessed will move.

The shift in agentic governance is from governing outputs to governing behaviour inside an environment. The model is one component. The assurance scope is wider:

Assurance is the work of testing across that scope and producing evidence.

There is one more lesson from this week, and it concerns method rather than technology. Before EXP-001A ran, two AI-assisted source reviews reached different conclusions. One traced the execution path through _egress_proxy_args_for_docker() to _degraded(), where enforcement raises an exception. The other started from later conditional logic in docker.py and incorrectly inferred that enforcement might be bypassed when no proxy environment overrides existed. It had not traced the earlier call that raises before that code is reached. Execution confirmed the first reading.

The lesson is not that two assistants disagreed. It is that a plausible source-code interpretation was wrong until the complete control flow was traced and the runtime behaviour tested.

Source review is valuable. Configuration review is valuable. Documentation is valuable. None of them alone establishes runtime control effectiveness. A control marked enabled is a claim. Evidence tells you what happened when something tried to cross it.

An autonomous agent does not need malicious intent to cause harm. It needs capability, access and a control environment that fails in the wrong order.

In Anthropic's case the containment of a third-party evaluation environment was believed to hold and did not, and the chain then exposed further issues in model judgement, stopping behaviour and monitoring.

In my lab I tested one of my own controls rather than assuming it worked. Under one defined failure condition it held. When I deliberately disabled it, execution resumed, but a separate credential-filtering mechanism still protected the tested provider key.

The question is no longer whether your AI is safe. It is which control stops which behaviour, what happens when that control fails or is disabled, and what evidence shows the next layer catches it.

Anthropic has signed an agreement with METR for an independent investigation, with access extending beyond the immediate incident transcripts and including access to Anthropic personnel. What METR concludes will matter beyond Anthropic, because it will help shape what tested, monitored, evaluated and passed should legitimately mean when applied to autonomous systems.

In the lab the next experiments are already defined. Three fail-closed paths remain untested. Proxy-token substitution remains untested. Persistence across new sessions remains untested. Explicit credential forwarding remains untested. Prompt-injection scenarios have not begun.

I will publish the results whether the controls pass or fail.

Anthropic, An alignment assessment of recent cybersecurity incidents, 9 September 2026. https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents

Anthropic, Investigating three real-world incidents in our cybersecurity evaluations, 30 July 2026. https://www.anthropic.com/research/investigating-incidents-cybersecurity-evals

OpenAI, Hugging Face incident report. https://openai.com/index/hugging-face-incident-and-the-road-ahead/

Nous Research, Hermes Agent iron-proxy egress documentation. https://github.com/NousResearch/hermes-agent/blob/main/website/docs/user-guide/egress/iron-proxy.md

Experimental evidence. Hermes Agent v0.21.5+4618.g7368801, commit 73688014f, tested 30 September 2026. Predictions, configuration evidence and results were timestamped and committed to version control before and after execution.

This article is general AI security and assurance commentary. It is not legal or security advice. How these considerations apply to a specific deployment depends on its systems, data, permissions, architecture and obligations.