Reading collection

How to Review AI-Generated Code Before Merging

Use a risk-based checklist to review code the author cannot yet explain. Decide when local AI review, GitHub PR review, or deterministic CI can help.

Editorial update

Decision in brief

Require a human owner to explain the behavior, affected dependencies, failure modes, and recovery path for every material change. Use deterministic tests for known invariants and keep automated review advisory. If nobody can explain a consequential code path or prove it behaves as intended, split, rewrite, or hold the patch.

Explain the change, test known invariants, review failure and recovery, and make a human decision to merge or hold.
Keep automated findings advisory and require an owner to verify consequential changes.

AI-generated code review checklist by change risk

Begin with the issue and expected behavior. Ask the author to show what changed, where execution enters the code, what state can change, and how the old behavior can be restored. Compare this description with the actual diff. Review added dependencies, configuration, data handling, interfaces, and tests that changed alongside implementation.

Use one independently specified regression test. Where practical, run it on the baseline and the candidate patch so you can see that it detects the original failure. Re-read the assertions when the agent authored both code and tests. Give higher-risk changes an owner who knows the subsystem and a tested recovery path before rollout.

Change or review needReview workflowEvidence before proceeding
Repeatable rule or known invariantDeterministic CI such as a focused test, type check, or linterAn independent check demonstrates the expected behavior and guards against regression.
Author needs private feedback before pushingLocal review of the staged diff where repository data policy permitsA human verifies each consequential finding and checks which context the tool received.
Team needs shared pull request feedbackTrial an already approved Copilot or Codex integration, or a dedicated bot such as CodeRabbitReviewers can reproduce useful findings, and the pilot bounds permissions, noise, and billing.
The patch changes permissions, persistence, or an unfamiliar critical pathSubsystem-owner review with staging and rollbackThe owner explains failure cases, verifies required checks, and rehearses recovery.

What two HN discussions reveal about review effort

In the Ask HN discussion about shipping code people do not understand, cadamsdotcom argued for tests observed failing before implementation. Use this as a review check. Ask for the smallest independent failing case that explains why the change is needed and confirm the reviewer can replay it.

In a separate review thread, ocharles reported finding private pre-review worthwhile. ohans reported finding automatic PR comments noisy in the same discussion. Evaluate author-side and team-visible feedback separately. A suggestion that helps an author may still interrupt every reviewer when posted to a shared queue.

Pilot one review workflow before changing merge policy

  1. Choose eight closed, sanitized, or synthetic changes. Include clean diffs and benign known regressions in formatting, state transitions, or expected test behavior. Have a maintainer write the expected findings and mark which cases are merge-blocking.
  2. Run the existing human and CI process, then run one advisory tool on the same commits with approved repository scope. For local tools, verify what context is sent to the provider. For PR integrations, record trigger, feedback visibility, repository permissions, and charges.
  3. Count reproducible findings, missed blocking cases, false or duplicate comments, and total human triage minutes. On a sample patch, ask an independent reviewer to explain the changed path and replay an acceptance test and the rollback.
  4. Keep the tool advisory and widen its use only if the pilot retains required human approval and all designated blocking regressions are caught by the combined process. Continue only if review effort or useful coverage improves. Stop on a missed blocker, unapproved access, or unclear ownership of rollback.

Related articles