Shipping AI-written code you don't understand: what the evidence says about 'comprehension debt', and what teams are doing about it

Shipping AI-written code you don't understand has become one of the most discussed problems in software teams that use coding agents. On October 19, 2026, an Ask HN post titled "How to deal with shipping AI-written code you don't understand?" drew hundreds of replies. Practitioners call the worry "comprehension debt": the gap between how much code a team ships and how much of it anyone understands. This analysis sets out what has been measured, what is still anecdote, and what official tools and open-source projects require before AI-written code ships.

This analysis is based on the Harvard working paper by Fiona Chen and James Stratton, METR's developer-productivity studies, Anthropic's October 2026 posts on how the Claude Code team reviews AI-written code and on employment at Anthropic, the Stack Overflow 2026 Developer Survey, Google's 2025 DORA report, project policies and official tool documentation, checked October 20, 2026. Hacker News comments are community reports (Tier 3) and are labeled as such.

The short version

• Measured: at 718 firms, coding agents brought more code and more pull requests, but review time rose 49% and resolved issues didn't change significantly (Chen and Stratton, August 2026).
• Measured, but small or dated: METR's 2025 randomized trial found experienced developers took 19% longer with AI while believing they were 20% faster. METR said in February 2026 that the finding is out of date and its newer data is unreliable.
• Vendor-reported: Anthropic says Claude Code's revert share fell by more than 40% while merged pull requests more than doubled, and that its engineers don't keep a complete mental model of the codebase.
• Surveyed: developers mostly trust AI output only when they can check it. Just 6.6% of Stack Overflow's 2026 respondents trust it for important decisions.
• Not measured anywhere we found: how much AI-written code developers fail to understand, and whether that leads to more production incidents.

What are developers actually asking?

The Ask HN post, submitted by the user rzk, quotes a LinkedIn post by an engineer who says AI-generated code "all feels like a black box": he understands the high-level design but not every detail. These replies are community comments, not verified claims, but they show where practitioners disagree:

• Design first, delegate implementation. "I don't let the LLM design the code at all, I let it implement a design I have created," wrote one commenter (bitwizeshift), whose team now holds joint design reviews.
• Small changes and heavy testing. A compiler developer (spacedcowboy) described "a long planning session" and small check-ins, while admitting "I couldn't say I'd reviewed everything."
• Tests aren't enough. Others replied that "Testing doesn't verify the design" and that LLM-written tests make weak guardrails.
• Rubber-stamping. "I just approve PRs, because I have to. I don't know what else to do," wrote another (birdsongs), describing thousands of changed lines a week.

What does the measured evidence say?

Last verified: October 20, 2026

Two patterns stand out. First, the strongest outside evidence is about review load, not understanding: more code arriving, reviewers taking longer and no measured rise in finished work. Chen and Stratton infer a higher bug rate from review outcomes but don't measure bugs. Second, self-reports are unreliable here. METR's 2025 trial found a 39-point gap between how fast developers felt and how fast they were, which matters for any team judging its own comprehension by how confident it feels.

What changed after firms adopted coding agents

change after agent adoption, 718 firms (working paper)

Lines of code per worker+30%

Commits+20%

Pull requests+23%

Review comments per PR+35%

Review time+49%

Resolved Jira issuesNot significant

Show the data

Measure	Change
Lines of code per worker	+30%
Commits	+20%
Pull requests	+23%
Review comments per PR	+35%
Review time	+49%
Resolved Jira issues	Not significant

What do the real-world cases show?

PostgreSQL 19. In an October 9 post, "The reverts will continue until morale improves," committer Peter Eisentraut said about 8 to 10 significant features were reverted this cycle. He argued that with LLM-assisted review, defects "can get found much faster," leaving developers "staring at a list of like forty defect reports" (an illustration, not a count). He also says the situation "is not due to 'vibe coding'," because the code predates it. Here AI review exposed problems in human-written code faster than people could fix them (our report).

QGIS and other bans. QGIS approved a flat ban on AI-, LLM- and agent-assisted code on October 7, 2026, joining Gentoo, NetBSD, QEMU and Servo, which cite reviewer time, copyright and quality (our analysis).

Linus Torvalds. The counterexample is expert-directed use. Torvalds' October 10 kernel patches say Claude Code wrote the code, tests and first draft "under my direction," with his own sign-off on each patch and a list of ten known problems (our report). That fits the "design first, then delegate" camp, and the argument in the "AI is a tool" debate that results depend on how closely the person directs the work.

What review rules do the official tools and projects document?

Last verified: October 20, 2026

The common thread is accountability, not comprehension. LLVM's "able to answer questions" rule comes closest to testing understanding. Most rules require a named human to approve and take responsibility, without checking what that person understood.

What are teams doing about it?

The practices in the sources above fall into four groups:

1. Gate by risk. Anthropic sets human review by what a change touches, not who wrote it, and pairs that with fast reverts, feature flags and staged releases. Its post concedes "Some mistakes will still get through."
2. Explain on demand. Anthropic asks: "If machines do most of the reading, how does anyone learn the code?" Its answer is that "Nobody on the team keeps a complete mental model of Claude Code in their head, and we don't think anyone needs to." When someone needs to understand a part of the code, they ask Claude to write "an explainer for exactly that slice."
3. Make the author prove understanding. LLVM's rule is one form. A tool shown on Hacker News on October 18, Sphinx, is another: a Git hook, MIT-licensed, that asks the developer a question about what a diff does when things go wrong before a push to a protected branch. Its site says skips via sphinx yolo are logged. Commenters were split; one called it "the wrong direction" and urged developers to learn the code instead. It's an example of the approach, not a measured fix.
4. Keep humans on design. Hacker News commenters describe owning the design and abstractions and delegating only implementation, plus small changes and joint design reviews. These are community reports.

Unattended agents raise the stakes, since nobody may read a change before it runs (our analysis).

What the evidence supports, and what it doesn't

Supported: coding agents increase code volume faster than review capacity at the firms Chen and Stratton studied. Review became the bottleneck, and finished work didn't measurably rise. DORA's 2025 survey points the same way on stability. Developers' trust is conditional on verification.

Mixed: whether AI-written code is worse. Anthropic reports no difference in revert or removal rates for its own engineers, and a falling revert share on Claude Code. These are vendor numbers about a company with unusual tooling and token budgets, and Anthropic itself notes that AI errors may be harder to spot. PostgreSQL shows AI review can raise the quality bar, not lower it.

Not established: that developers understand less of the code they ship than before, or that this causes more incidents. No study we found measures comprehension directly. "Comprehension debt" is a useful name for a real worry, not yet a measured quantity.

Competing explanations: the Ask HN anxiety may reflect review overload (Chen and Stratton's result) or pressure to ship faster, which several commenters described, rather than inherently opaque code. Those call for different fixes: more review capacity and smaller changes, or slower delivery targets.

What's confirmed, reported and uncertain

• Confirmed (primary documents): the Chen and Stratton results; METR's 2025 and 2026 findings and caveats; Stack Overflow's 2026 trust figures; DORA's 2025 trust and stability findings; the GitHub, Claude Code, LLVM, kernel and QGIS rules quoted above.
• Vendor-reported: Anthropic's 90% figure, revert-share drop and no-difference result for adopters.
• Community reports: the Ask HN practices and the reaction to Sphinx.
• Uncertain: whether comprehension gaps lead to more incidents; whether Anthropic's results carry over to teams without its tooling; how DORA's 2026 findings, if published, compare.