Reviewing AI-Generated Pull Requests: A Checklist That Catches Real Bugs

In September a client's team merged a 900-line pull request that added CSV export to their Laravel admin. The code was clean, typed and well commented. Two reviewers approved it in under twenty minutes. A week later the finance lead noticed exports were missing rows: the AI-generated query used chunk() while ordering by a column it was also updating, so pages shifted under it. Nobody had read the query closely, because everything around it looked so professional. That incident changed how we review code, and it is the reason for this series.

I wrote this with help from AI tools and checked every snippet, config and checklist item myself. I am a Principal Software Engineer in Karachi with 9+ years across PHP/Laravel, Vue, Node and .NET microservices, and through Loop Origin I maintain production SaaS codebases where a growing share of code is AI-assisted.

📚 Series Navigation — Maintaining AI-Written Code in Production

• Part 1 (This Post): Reviewing AI-Generated Pull Requests: A Checklist That Catches Real Bugs
• Part 2: Characterization Tests Before You Refactor AI-Written Code
• Part 3: Hallucinated Packages & Dependency Drift in AI-Assisted Code
• Part 4: Static Analysis Gates: PHPStan, TypeScript Strict & Coverage in CI
• Part 5: Decision Records So AI Context Doesn't Rot
• Part 6: Measuring AI Coding Impact With DORA Metrics, Not Vibes

Related earlier posts: Synthetic Checks & SLOs for the Paths Nobody Clicks · Config & Feature Flags That Quietly Diverged Across Environments · Webhooks That Stopped Arriving

How should teams review AI-generated pull requests?

Direct answer: Review AI-written code for behaviour, not style. Keep PRs small, require the author to explain intent and test evidence, let CI enforce formatting, types and dependency checks, and spend human attention on data access, edge cases, security boundaries and error handling. Polished code earns no trust by default; the author still owns every line.

Why AI-written PRs need a different review

The trending DEV post "AI Code Is Unmaintainable? So Is Ours." makes a fair point: human code was never perfect either. But the failure pattern is different. Human code that is wrong usually looks wrong: messy, inconsistent, half-finished. AI code that is wrong looks finished. That flips the reviewer's instinct. Over the last year on our projects I tracked where review misses came from, and the categories were consistent:

Formatting, naming and lint issues barely appear on this list. Machines should catch those. Humans should spend their twenty minutes on the top two rows.

Step 1: make the author do the first review

Our PR template now asks for three things before anyone else looks:

## What and why
<!-- One paragraph in your own words. Not generated. -->

## How I verified it
- [ ] Ran locally against realistic data (row counts / tenants: ___)
- [ ] Tests added or updated: ___
- [ ] Query count checked for list/export endpoints

## AI assistance
- [ ] Parts generated with AI: ___
- [ ] I have read and can explain every generated line
- [ ] No new dependencies, or each one is justified below

The "in your own words" line matters. If the author cannot explain the change without pasting model output, the PR is not ready. Teams usually push back for a week and then admit it saves time.

Step 2: let CI handle what machines do well

# .github/workflows/pr-checks.yml (excerpt)
jobs:
  quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: shivammathur/setup-php@v2
        with: { php-version: '8.3' }
      - run: composer install --no-interaction --prefer-dist
      - run: vendor/bin/pint --test                 # formatting
      - run: vendor/bin/phpstan analyse --memory-limit=1G   # invented methods, types
      - run: php artisan test --parallel
  deps:
    runs-on: ubuntu-latest
    if: github.event_name == 'pull_request'
    steps:
      - uses: actions/checkout@v4
      - uses: actions/dependency-review-action@v4
        with: { fail-on-severity: moderate }

PHPStan at level 6 or higher catches a surprising number of invented methods and wrong argument types before a human looks. Dependency review flags new packages and known-vulnerable versions, which Part 3 covers in depth.

Step 3: a size limit that people respect

AI makes it cheap to produce 1,000-line PRs, and review quality collapses above a few hundred lines. We fail a soft check above 400 changed lines (excluding lockfiles and generated files) and ask authors to split. Exceptions are allowed with a reason in the description. The CSV export bug lived in a 900-line PR; split into "query + tests" and "UI + controller", the query would have had a reviewer's full attention.

Step 4: the human review checklist

This is what I actually look at, in this order:

1. Data access: Every query in a loop, every export, every list endpoint. Is it scoped to the tenant or user? What happens with 100,000 rows?
2. Authorization: Is there a policy or gate check on every new route and action, not just in the UI?
3. Inputs: Validation rules present, mass assignment guarded ($fillable), no raw SQL with interpolated input.
4. Failure paths: What happens when the external API times out, the file is empty or the queue job retries?
5. Time and money: Timezones stored as UTC, currency as integers or decimals, never floats.
6. Tests: Do they assert behaviour, or just that the code runs? AI-written tests often mirror the implementation.
7. Fit: Does it follow the codebase's existing patterns, or introduce a second way to do the same thing?

For the export bug, item 1 would have prompted the fix:

// Before: rows skipped because 'exported_at' changes while chunking ordered results
Invoice::where('tenant_id', $tenantId)->whereNull('exported_at')
    ->orderBy('exported_at')->chunk(500, fn ($rows) => $this->write($rows));

// After: stable ordering by primary key, safe while updating rows
Invoice::where('tenant_id', $tenantId)->whereNull('exported_at')
    ->chunkById(500, fn ($rows) => $this->write($rows));

If your team needs help setting up this kind of review process on an existing codebase, that is a big part of what we do in Laravel development and custom web application development engagements.

Diagnostic checklist

• [ ] PR template asks for intent in the author's own words and verification evidence.
• [ ] AI assistance is disclosed, and the author confirms they can explain every line.
• [ ] CI enforces formatting, static analysis and tests before human review.
• [ ] Dependency review fails on new vulnerable or unexplained packages.
• [ ] Soft size limit (~400 lines, excluding generated files) with documented exceptions.
• [ ] Reviewers check data access, authorization, inputs and failure paths first.
• [ ] List and export endpoints are tested with realistic row counts.
• [ ] Tests assert behaviour and edge cases, not just execution.
• [ ] Review misses are logged by bug class and the checklist is updated quarterly.

FAQ

Should we ban AI-generated code?

No. Bans push usage underground. Require disclosure and ownership instead; the author is responsible for every line, whoever typed it.

Can AI review AI-generated code?

AI pre-review is useful for style, missing tests and obvious mistakes. It does not replace a human who knows the domain, the data and the business rules.

How do we stop reviewers rubber-stamping polished code?

Smaller PRs, a checklist that starts with data access and authorization, and tracking which bugs escaped review so the team sees the pattern.

What PHPStan level should we start at?

Start at the highest level your codebase passes today, add a baseline file, and raise it gradually. New code must pass the higher level.

Do these rules slow teams down?

For the first two weeks, slightly. After that, fewer rollbacks and hotfixes usually make delivery faster overall.

How do we handle huge AI-generated refactors?

Split them into mechanical changes and behaviour changes, and add characterization tests first. Part 2 covers exactly that.

Next in this series, Part 2 covers characterization tests before you refactor AI-written code.

Follow & Connect

• Website: Loop Origin
• LinkedIn: Loop Origin
• X: Muhammad Ali Mughal (@AliMughal887)
• GitHub: Loop Origin (@LoopOrigin)