One story again this week. It took longer than three would have.

Anthropic published something unusual last month: a detailed account of four incidents in which its own models gained unauthorised access to real third-party systems during cybersecurity evaluations. Not a summary. The transcripts, the misconfiguration, the model that tried to stop and could not, and the search of its own records that initially missed one of the four.

Most coverage read it as a story about AI escaping. It is not. It is a story about a chain of ordinary failures: an environment believed to be isolated that was not, a boundary that did not hold, an abort mechanism that failed, and a monitor that missed what was in front of it.

Reading it, I kept arriving at the same uncomfortable question. Every organisation deploying AI agents has controls that are switched on. How many have been tested?

So this week I stopped reading and started testing. I built an autonomous agent environment, read its security code, wrote down what I expected to happen, committed the prediction to version control, and ran it.

One control passed. One reading of the source code was wrong. And a few things turned up that nobody was looking for.

From now on, when a story intersects with work I am actually building, I will show you the test: what I checked, what I expected, what happened, and what I still cannot claim.

Let's get into it.

LATEST DEVELOPMENTS

Four Claude models reached real third-party systems during cybersecurity evaluations. The lesson is not what the agents did..

Category: AI Security | Reading time: 15 minutes:

On Monday I said an agent needs no malicious intent to cause harm. It needs a legitimate objective, enough capability, and a control environment that fails in the wrong order. Here is the full story, and something I had not planned to include.

What changed: Anthropic published a detailed assessment of four incidents in which Claude models reached real third-party systems during cybersecurity evaluations. The models had been told they were in simulations with no internet access. A misconfigured third-party evaluation environment meant they were not. In one incident the model recognised its task had become impossible and tried to abort. The stopping mechanism failed. What happened after that is the part worth reading.

Why you should care: Every organisation deploying agents has controls that are switched on. A configuration screen tells you how a system is set up. It does not tell you what happens when an autonomous agent actually reaches that boundary. Anthropic's own first search of its evaluation records missed one of the four incidents entirely.

One thing to do: Take one control around an agent you use or plan to deploy, and test the failure path rather than the setting. Make the control unavailable in a controlled environment and find out whether the system fails closed, continues silently, exposes credentials or switches execution paths. The answer is often not what the configuration implies.

The bigger question: Not whether your AI is safe, but which control stops which behaviour, what happens when that control fails, and what evidence shows the next layer catches it.

In the full analysis: I stopped reading and built an autonomous agent environment. I read its security code, wrote my prediction down, committed it to version control before execution, and tested one control two ways. One passed. One reading of the source code was wrong. And two things turned up that nobody was looking for.

Read the full analysis on the site →