Engineering problem

Claude Code Review: Skills and Automated PR Review

Compare local Claude Code review, reusable review skills, and Anthropic’s GitHub Code Review. Use a sample of known regressions to measure which findings are worth the reviewer’s time.

Editorial update

Decision in brief

Use Claude Code as a second reviewer for concrete regressions you can verify. Start with a local review and save it as a skill once the checklist repeats. Consider Anthropic’s managed GitHub Code Review if the team needs repeatable PR coverage and the extra comments improve outcomes. Keep CI and named human reviewers responsible for the merge.

Verify code review: Flag a regression. Run the check. Human decides.
Keep CI and a named human reviewer responsible for the merge.

Pick the review surface by what has to change

A Claude Code review skill standardizes the instructions for a local session: what to read, which commands are allowed, and what a useful finding looks like. Anthropic’s managed Code Review is a separate GitHub product that sends findings back to the pull request. GitHub Actions is another option when the team needs a custom trigger or workflow. Decide which surface you want before comparing its setup, permissions, or bill.

Review needStart withWhat would justify changing it
Preflight on an unpushed branchGive a fresh local session the base commit and ask for file and line findings with an explanation of impact.Save a skill once the same checklist and evidence format recur across several reviews.
An invariant a deterministic check can enforcePut it in CI: e.g. run a fixture that confirms an export keeps the original row count.Use the agent for surrounding behavior CI cannot yet explain; keep the automated invariant.
Consistent coverage across GitHub PRsTry managed Code Review on a small set of repositories with a human triage owner.Expand if it catches real regressions earlier without growing duplicate or false-positive comments.
Team-specific triggers or reportsConsider a scoped GitHub Action with explicit triggers and an owner.Use a local review if workflow secrets, repo permissions, or support overhead outweigh the benefit.

What went differently in Hacker News discussions

On a review-skill thread, rgambee pointed out that many recurring anti-patterns could be prevented while writing the patch by moving their guidance into CLAUDE.md. That suggests a three-way split: put repeatable conventions in project instructions, mechanically checkable conditions in CI, and context-dependent questions in the review skill.

In the discussion of Anthropic’s review practice, vavkamil argued that dismissing a critical finding as a false positive should involve a security owner. An improved revert rate, by itself, would not settle that question. In a separate self-test, piotrgrudzien reported that a heavier review found a few more seeded issues than a simpler review but took substantially longer. The test is a reason to track cost and time next to findings; its small, selected sample cannot establish a general win.

Run a paired review against five benign fixture patches

Use five private or disposable patches from one repo: three that each break a known behavior, such as changing the fallback for empty input or changing a CSV header, and two known-good changes. A separate teammate holds the answer key. Decide what counts as one finding before running anything.

  1. Record a common base and run the repo’s existing tests. Use a fresh session per patch and give it the same scope, a read-only checkout or equivalent restrictions, and the same review prompt.
  2. Ask for findings only: file and line, broken behavior, the input that demonstrates it, and why the change caused it. Ask reviewers to flag uncertainty explicitly. Use an independent passing/failing test to check each claim.
  3. Compare the plain prompt with the same workflow saved as a skill, or with the managed GitHub review, one change at a time. Record true regressions, misses, false positives, duplicate comments, and human triage minutes per patch.
  4. Pilot on real PRs only if the method finds more confirmed regressions or saves triage time at comparable recall. Keep CI and human sign-off in place; route consequential or uncertain findings to the responsible maintainer.

Related articles