Anthropic's Addy Osmani Warns AI Agents Can Outrun Human Review

Addy Osmani, an Anthropic engineer who works on Claude Code, warned Nov. 3 that AI agents can produce software faster than teams can verify it, leaving developers responsible for decisions they cannot explain.

Osmani described a gradual shift in how people use agents. A developer may initially delegate a task while checking the result, then begin treating each answer as the basis for the next task. Eventually, the developer may accept an answer without thinking to question it. Osmani described that final stage as “cognitive surrender,” a term coined by researchers Steven Shaw and Gideon Nave.

The problem can surface well after an agent finishes its work. A project may have orderly code and passing tests, yet the team may discover during a later change that no one understands why the system was designed as it was. Osmani called that accumulating loss of understanding “cognitive debt.”

That does not mean engineers must keep every detail of a codebase in mind. Osmani argued that they need to recognize what they have not checked and be able to rebuild their understanding when a decision matters.

Tests also have limits. An agent can write a feature and its tests around the same mistaken assumption, allowing the tests to pass without establishing that the feature meets its intended goal. Developers therefore need to ask what each test checks and what remains untested.

Osmani described the gap between an agent's output and a person's capacity to review it as an “orchestration tax.” Adding more agents may increase the amount of work awaiting review without increasing the team's ability to approve it responsibly. He suggested stronger automated checks or a slower pace of output when review becomes the constraint.

He also warned about “intent debt”: choices an agent makes when a task leaves its goal, limits or design reasoning unstated. Those choices can be hard to spot in a finished result that appears complete.

Osmani urged developers to define success before assigning a task, inspect the commands an agent ran and examine evidence such as test records. He recommended forming an initial view before asking an agent for an answer, then looking for reasons the answer might be wrong.