Video summary
Anthropic Stopped Reading Every Line of AI Code. Here’s What Replaced It.
Main summary
Key takeaways
Summary
Anthropic’s Claude Code team moved away from treating “a human read every line” as the measure of a good code review. As code changes rose from roughly 2,500 per day in May to more than 30,000 in September, manual line-by-line review could not scale. The team reported that merged pull requests more than doubled, while the share later rolled back fell by over 40%.
Instead, the team uses a set of checks that starts before a pull request and continues through release. The guiding question is:
What could go wrong with this change, and which check—automated or human—is best suited to catch it?
The seven layers
- Verify changes before opening a pull request. Agents run tests, builds, or visual checks and show the results. A second agent reviews the work in a fresh context. Recurring review feedback can be turned into lint rules or project guidance files such as
CLAUDE.mdandREVIEW.md. - Run AI review on every pull request. Multiple agents look for bugs, while a separate “jury” evaluates their findings to filter out unsupported or low-value comments. Findings are prioritized, supported with file-and-line evidence, and must be fixed or answered.
- Use a separate security review that can block a merge.
- Require CI checks. Linting, type checks, tests across platforms, and builds must pass. Pull-request templates should explain how a change was tested and how it can be undone.
- Scale human approvals to risk. Low-risk changes may need no human approval; ordinary or sensitive changes need one; and high-risk changes need two. Risk is classified from the change, AI approval does not count as human approval, and the person who directed an AI agent remains accountable.
- Merge safely and release gradually. A merge queue reruns checks against the latest code. New behavior can be released behind flags, first to employees and then to broader channels.
- Detect and recover quickly. Feature flags allow behavior to be disabled without a new deployment. On-call rotations and incident reviews examine which layer should have caught the problem.
Keeping AI review from spiraling
The team encountered a pull request that went through more than 50 rounds of AI review and fixes. Many later comments repeated issues identified in the first review, and agents could turn minor comments into extra commits and review cycles.
The team adjusted the process by limiting review rounds, asking the reviewer to reconsider the overall approach after several passes, focusing later reviews on new changes, addressing whole classes of bugs, and preventing repeated findings.
After these changes:
- The share of changes passing after one round rose from 65% to 78%.
- Reviews requiring five or more rounds fell from nearly 9% to about 1%.
- Average rounds to pass dropped from 2.08 to 1.32.
The team also evaluates reviewers against confirmed incidents and reversions, and tracks whether developers fix, reject, or dismiss comments to measure review noise.
What remains for people
Humans focus on decisions machines are less equipped to make: whether to build something, how to handle high-risk areas such as billing or CI, architectural choices, interpreting evidence, and owning the outcome.
The video also recommends asking an agent to explain the relevant module before making a change, so developers can continue learning the codebase. Whoever lands a change remains responsible for it, regardless of who—or what—wrote it.
Practical guide
The video recommends starting with these steps:
- Give the coding agent checks it can run, documented in a short
CLAUDE.md. - Run code and security reviews on changes before opening a pull request.
- Add a
REVIEW.mdand move recurring review comments into it. - Add testing, risk, and rollback instructions to the pull-request template.
- Define risk tiers, starting conservatively and loosening them based on evidence.
- Cap review rounds at three.
- Turn every confirmed reversion into a test case for the reviewer.
Main speakers and sources: The video’s narrator; Anthropic Claude Code team engineers, including Claude Code creator Boris Cherny; and Anthropic’s reported internal review practices and metrics.
Rate this summary
Your feedback will help improve summaries.
Improve this summary
Reprocess with a stronger model when the summary feels incomplete or inaccurate.
Translate summary in another language
Ask questions to this video
Chat for follow-up questions, clarifications, and source-backed answers.