Radical transparency. We build AI agents. We use AI agents in our own workflow. One of them failed in a way that taught us something important. Here is the story.
What we built
We deployed an autonomous code review agent in our development pipeline. The agent received pull requests, reviewed the code for security issues, performance problems, and conformance to our code standards, and left comments. We gave it the ability to approve PRs if it found no significant issues, and to request changes if it did.
What went wrong
Over three weeks, the agent approved 47 pull requests. Four of those PRs contained issues that should have been caught — two security-relevant and two performance-relevant. The agent missed them. When we investigated why, the pattern was clear: the agent was good at identifying issues that matched patterns it had been explicitly trained to look for, and poor at identifying novel issues that required contextual reasoning about the specific codebase.
The specific failure mode
In one case, a PR introduced a query that would work correctly in isolation but would cause an N+1 performance problem when called in the context of an existing endpoint. The agent reviewed the PR in isolation, saw a correctly written query, and approved it. A human reviewer with context about the calling code would have caught the problem immediately.
What we changed
We removed the agent's ability to approve PRs autonomously. It still reviews, comments, and flags issues — but all PRs now require human approval. The agent's comments are useful and save review time; the autonomous approval was where the risk was concentrated. We also added a requirement that the agent review the calling context, not just the changed lines — which caught a similar issue within the first week of the new configuration.
The lesson
Autonomous approval is a Human Gate decision. We had designed the agent with the wrong scope of autonomy for the risk level. Code review approval is consequential and context-dependent — exactly the conditions that require a human gate. We knew this principle; we did not apply it correctly in this case. Now we do.