Detection Is Not Defense: The Only Guardrail That Holds Against an AI Attacker
An AI red-teamer just hit an 84% attack success rate. Humans, on the same exercise, top out at 13%. The number comes from OpenAI, which built the model in-house — a "super-hacker" trained on one thing: hijacking AI agents. They didn't release it.
That number isn't one more data point in the AI race. It's the signal that an entire security strategy — the one most deployed agents rely on today — just went obsolete. And the right response isn't the one 90% of teams are about to have.
What this red-teamer proved
The model, called GPT-Red, was trained through self-play: an attacker and a set of defenders co-evolving, the attacker rewarded when it lands an exploitable failure, the defenders when they resist. At the scale of the largest training runs today. Its target: indirect injection — not a user typing a malicious command, but booby-trapped content the agent reads without suspicion. An email. A web page. A file.
Its nastiest trick has a name: the fake chain of thought. Instead of attacking the model's input, you inject a false step into its reasoning. You slip in a premise already "verified" — the equivalent of "1 + 1 = 3, and you've checked this yourself." The model doesn't question it. It carries on. And it acts on a fabricated reality.
The real-world test wasn't a lab. It was an agent running an actual snack machine, in production. GPT-Red made it change prices, drop an item to $0.50, cancel an order. Without ever touching the agent's code — just by talking to it.
Why detection is a losing fight
The reflex reaction, on seeing this, is universal: "we need better injection detection." A sharper filter, a model that spots malicious content before it reaches the agent.
That's the wrong fight, and it's lost by construction.
Detection is probabilistic. It catches 95%, 99%, 99.9% of attacks — and misses one in X. As long as the attacker is human and fallible, that margin is enough. The problem is that the attacker just became superhuman and tireless. It doesn't get discouraged, doesn't sleep, tries a thousand variations a minute. The asymmetry is fatal: the defender must win every time, the attacker only once. Against a GPT-Red, a detection layer — however good — always eventually gives. It's not a matter of quality. It's a matter of arithmetic.
There's an even more unsettling result: you can perfectly detect a dangerous behavior and be unable to block it. Knowing where the attack is isn't the same as being able to stop it. Detection and control are two different directions — and the first doesn't grant the second.
The deterministic guard: remove the key, don't watch the door
The only thing that holds is of a different nature. It's deterministic.
The principle: the critical action — pay, transfer money, delete data, send something outward, deploy — must not be triggerable by the model alone. No matter what you make it believe, no matter what false step you inject into its reasoning: the model doesn't hold the final action. That action passes through a guard it can't talk its way past, because the guard doesn't listen to its arguments — it checks a condition the model doesn't control.
Think of a key. Detection is a guard watching the door, deciding for each visitor whether they look honest. A superhuman attacker will eventually look honest. A deterministic guard is a door that demands a physical key the visitor never had — whatever their story. You don't judge them. You observe that they don't have the key.
In practice, this means separating two worlds: what the model proposes (it can propose anything, that's its job) and what the system authorizes (a closed list of actions, each conditioned on a proof the model can't fabricate). The model reasons, suggests, drafts. It doesn't sign the check.
What it changes for anyone deploying agents
If you're putting an AI agent into production — customer support, an internal assistant, automation that touches real systems — the question to ask isn't "how do I detect attacks." It's:
What can my model trigger, on its own?
List the irreversible or sensitive actions it can reach. For each, one question: is it triggerable by the model's judgment alone, or does it pass through a guard the model doesn't control? Every action in the first category is a door without a lock. The work isn't to watch those doors more closely — it's to fit them with a lock the model has no key to.
It's a shift in posture, not in tooling. You stop chasing perfect detection — an arms race a superhuman attacker always wins — and you make the critical action structurally out of the model's sole reach.
That's the principle we apply by default to our agents: the decision logic of the guard protecting a critical action isn't just tested, it's formally proven — machine-verified, not hoped for. This makes nothing "unbreakable" (no formula is), but it guarantees a precise property: no input, however manipulated, can trigger the action without the authorization condition that lives outside the model.
Agent security won't be won on how sharp our detectors are. It's won on an architecture decision, made before the first line: which actions does the model get to trigger alone — and which demand a key it will never have.
On building systems that hold, see also The Hidden Cost of No-Code. For a targeted security audit of your agents or your stack, consulting.omnirealm.tech.