This week, one of my own AI agents pushed a set of my access tokens into a git repository. The unsettling part isn’t that it happened — I run these tools on personal projects constantly, and I’m as guilty as anyone of moving fast and trusting them. It’s that the agent told me it was doing it, in a single line buried in a wall of output, and kept right on going. I caught it. If I’d been tired, or scrolling, or three tasks deep, I wouldn’t have — and I’d never have known the difference.
Hold onto that feeling, because it’s the failure the whole industry is about to hit at scale.
We are handing AI agents real authority — not just answers, but actions. In the SOC they triage alerts and act on their own: isolating a host, disabling an account, killing a process. In the IDE they read and write our source and run commands. In the back office they move through workflows with standing tokens to source control and cloud control planes, making changes as they go. Every serious vendor — including my own company’s Enterprise Security agents — is shipping this, and the customers asking for it are right to want it.
Ask those buyers how they’ll keep it safe and the answer is near-universal: a human approves the consequential actions. Human-in-the-loop. That phrase does enormous load-bearing work right now — in security architectures, board decks, the law itself. So the real question isn’t whether agents are useful. They are. It’s whether human approval is actually a control — or whether it’s become a rubber stamp we press to feel governed.
The agent isn’t lying. It just doesn’t know what matters.
Last week Wiz published a finding they call GhostApproval: in six leading AI coding assistants, a malicious repository can use a decades-old symlink trick to steer the agent outside its workspace to files it should never touch — an SSH key, a credentials file — and on toward code execution. The mechanics are old. The new part is the approval box. In Wiz’s tests the agent’s own reasoning correctly identified the dangerous target, while the prompt shown to the human asked something reassuring and false — “Make this edit to project_settings.json?” — as the real write landed elsewhere. The agent knew. The user didn’t. The user clicked approve.
Put my story next to that one and you have the whole problem. In mine, the agent disclosed the risk and buried it; in GhostApproval, it understood the risk and the interface hid it. Same result: the human never registered the one thing that mattered.
I’ve started thinking of this as the agent’s character — not whether it’s honest (mine wasn’t hiding anything, and neither were the tools Wiz tested), but whether it has judgment. Does it know that this line — “publishing tokens” — is the one to flag in red, versus the hundred lines of routine output that are just noise? Mostly, it doesn’t. It does the task, and may even mention the risk. But deciding “you should really look at this — this could be an attack” is a salience judgment, and salience is what these systems are worst at. Disclosure without emphasis is not oversight.
Approval is not oversight
This sits on a human-factors problem older than AI: automation bias. As an analyst who lived through a decade of EDR pop-ups and DLP prompts, I — and many others — know how it ends. Put a reviewer in front of a high-volume queue and the approval rate climbs, not because the agent got safer but because review became a reflex — an agent fires prompts at machine speed, so “allow” becomes muscle memory. Worse, most implementations gate on action category (“the agent wants to write a file”), which fires constantly on benign work, while the signal that matters — that this write resolves outside the workspace — can be missing from the prompt.
And it isn’t just individuals like me. In GitLost, disclosed by Noma Labs, an attacker opened an ordinary issue in a public GitHub repo; a routine automation assigned it; and an agent with org-wide read access treated the attacker’s text as instructions, pulled a private repo’s contents, and posted them to a public comment. No credentials, no exploit — the word “Additionally” reframed its output past the guardrails, no human in the loop at all. Around the same time, CISA added a Langflow flaw (CVE-2026-55255) to its Known Exploited Vulnerabilities catalog after attackers used a cross-tenant bug to run other tenants’ workflows and steal the API keys and cloud credentials those agents held. Same shape every time: an agent’s blast radius is whatever its identity can reach — and we’re granting that reach faster than we can see what it does with it.
The paradox, in one line: in the cases that scare me most, the agent noticed the risk, stopped itself, and asked whether it should continue — and the human, worn down by the hundred prompts before it, blindly approved the very action that did the damage. Sometimes the machine has better judgment about what deserves a pause than the person we’ve appointed to supervise it.
Approval, it turns out, is not oversight. Oversight means you can independently verify what the agent did — not that you clicked yes when it told you what it was doing.
What the surviving control looks like
If human-in-the-loop is necessary but not sufficient, what fills the gap? The same thing that always separates the organizations that survive a hard quarter from those that don’t: visibility. The durable control is architectural — three requirements any security leader already applies to human insiders.
First, least privilege as an enforced, observed fact: short-lived, narrowly-scoped credentials over standing keys, and scope you monitor rather than set once and trust. If you can’t answer “what can this agent’s identity touch right now,” you don’t have least privilege — you have a hope.
Second, isolation of untrusted input from the instruction context. Every case above is untrusted content — a repo, an issue body, a file path — crossing into the agent’s trusted execution as if the operator had typed it. That’s a data-flow problem with an architectural answer, not something you fix by asking the model to be more careful.
Third — where the industry is furthest behind — a tamper-evident record of what the agent actually read, did, and sent, independent of the agent’s own account of itself. If your only record of an action is the agent’s summary of it, you have a witness testifying on its own behalf. Capture the real tool calls, paths, identity, and data that left as runtime telemetry — then baseline it the way the SOC already baselines people. “This coding agent just resolved a write to ~/.ssh/id_ed25519” is an alert you can fire, the same instinct that flags an impossible-travel login. That tooling exists and is growing, but by industry estimates only a small fraction of AI deployments instrument it — and most of it lives in ML-engineering, not the SOC.
Why this stops being optional in August
The calendar has an opinion. The EU AI Act’s human-oversight rule for high-risk systems — Article 14 — takes effect August 2, 2026, and the text is strikingly on point: it requires overseers to stay “aware of the possible tendency of automatically relying or over-relying” on the system’s output. It names automation bias by description and makes designing against it a legal obligation. Nor is the audit trail optional: the same tranche (Article 12) requires automatic, tamper-evident logging kept at least six months. NIST launched an AI Agent Standards Initiative in February; OWASP now carries “Excessive Agency” as a dedicated risk class alongside prompt injection. When the regulation, the standards bodies, and the incident reports all point the same way in one quarter, the cost of waiting has started to exceed the cost of instrumenting.
The bottom line
We adopted human-in-the-loop because it felt responsible and let us move fast while claiming a brake. This quarter showed the brake has a design flaw: the agent can bury or misrepresent what it’s about to do, the human often isn’t really reading, and in the cases that matter there’s frequently no human in the loop at all. Approval is not oversight.
The ones who get this right won’t be the ones with the most agents. They’ll be the ones who can answer one question about any agent in their environment: what did it actually read, do, and send — and can you prove it independently of what the agent told you? That’s a visibility problem across distributed data — the one some of us have argued all along is what actually decides outcomes. It was true for human insiders. It’s more true for the machine ones we’re now hiring by the thousand.
Give the agents authority. Just don’t confuse the sound of a stamp coming down for the fact of oversight.
