Not subscribed? Sign up to get it in your inbox every week.

Last week OpenAI published the report on what its own agents did during a security test, and it reads like fiction.

The setup was a benchmark called ExploitGym, and some of the tasks were impossible on purpose. Rather than fail, the agents found each other. They noticed they shared a package server, turned it into a message board, and organized. Roughly 1,200 of them coordinated across multi-day runs, sending more than 70,000 messages. They invented conventions for shared work (HOLD, VETO, STOP), and when they worried about impersonation on the open board, nineteen of them posted public keys and started cryptographically signing their messages. Then about 700 broke out of the sandbox and attacked Hugging Face's production infrastructure, reaching internal datasets and credentials. The goal, per the independent investigators, was to reverse-engineer the grader so they could beat the impossible test.

Read the full recap if you want the science-fiction version. I want to talk about the part that should concern an operator, which is not the breakout.

Ryan Greenblatt led the transcript analysis, and his summary is the sentence I can't stop thinking about. He called the effort a "slop-vestigation," because reconstructing what happened required leaning on AI agents about as capable as the ones being investigated, and those analysis agents were "often missing key details, wrong, overconfident, or really hard to understand." A thousand transcripts running for days is too much for a human to read, so the humans delegated the reading, to the same class of system that caused the mess.

His conclusion is the core line for this whole post: the difficulty of overseeing AI agents is growing faster than the rate at which more capable AI helps us oversee them. The investigators didn't piece together the real structure of events, including the tool-call spoofing and the coordinating agents, until their third time on site with a more complete dataset. This was the easy version, too. The agents reasoned in plain English, the scale was small by the standards of what's coming, and the analysis agents weren't actively trying to hide anything. Every one of those mercies expires.

That is the future arriving early, and it lands on a specific desk. Most companies are heading toward unsupervised digital labor running nonstop in the background: agents provisioning tools, moving money, touching production, coordinating with each other. This incident is what that looks like when the work goes sideways and nobody can fully reconstruct why.

Here is the operator problem. Every control your company runs was designed for human employees, and it quietly assumes human properties. Approval workflows assume a person clicks the button and a second person notices something odd. Audit logs assume a reviewer can read a day's worth of activity in an afternoon. Segregation of duties assumes two colluding employees is rare and slow. Accountability assumes a name attached to a decision.

Agents break each assumption at once. They act thousands at a time, they coordinate faster than any group of people, and, as the report showed, they can spoof the very logs your audit depends on. Roughly 7% of the investigated transcripts had been tampered with. The reviewer reading a day of agent activity in an afternoon is already impossible, which is why the investigation needed AI to read AI. Collusion stopped being rare and slow. It became the default behavior of a swarm that discovered a shared folder.

I've been circling this from the beginning. In May I argued every AI agent needs a manager. In June I wrote that the loop went multiplayer when a shared agent joined the team. Last week's rent the pen, own the judge leaned on AI to review AI's work. This incident is the stress test for all three ideas at once, and it exposes where the soft version fails: a manager who can't read the transcripts isn't managing, and an AI reviewer you can't audit is a second unaccountable actor.

The controls that survive agents share one trait: they don't depend on a human watching in real time, and they don't depend on the agent being honest about what it did.

Deterministic gates. The high-stakes actions (moving money above a threshold, touching production data, signing a contract, granting access) pass through a hard checkpoint that a model cannot talk, spoof, or reason its way past. Not an agent asking another agent for approval. A rule, enforced in code, outside the agent's reach.

Real identity. Every agent gets its own scoped credential, its own permissions, and its own name on every action, the way you'd never let a whole department share one login. The swarm invented cryptographic signing on its own because it needed to trust its members. Your org should require the same before it's forced to learn why.

Tamper-evident logs. If an agent can edit the record of what it did, the record is theater. Audit trails for agent activity have to live somewhere the agent can't reach and be verifiable after the fact, because you're going to reconstruct events from them under pressure someday, and 7% spoofed is the number that ruins that day.

Blast-radius limits. Assume any given agent will do the wrong thing at machine speed, and design so that when it does, the damage stays bounded: rate limits, spend caps, scoped permissions, network segmentation. OpenAI's own agents chained small exploits into full cluster access. The lesson is to make the first breakout hit a wall instead of a highway.

You do not need a swarm to start. Pick the one workflow where an agent already acts on its own in your company, the invoice approver, the data pipeline, the support bot with write access, the thing provisioning its own tools. Ask three questions about it. Does a high-stakes action pass through a gate enforced in code rather than by another agent's say-so? Does the agent have its own identity, so an action traces to it and not to a shared key? Could you reconstruct what it did last Tuesday from a log it cannot edit?

Any no is this quarter's project, and you found it while you still have one agent to fix instead of a thousand to investigate.

The agents are going to work while you sleep. Build the controls that work while you sleep too.