Nothing Should Merge on an AI's Word

Why AI-written code needs quality gates that fail closed, and what a good one checks.

AI coding agents are fast now. They can read a repository, plan a change, write it, and open a pull request in minutes. That speed is real, but it creates a quiet problem: the volume of changes grows faster than a team's ability to check them.

When an agent says "done," teams tend to take one of two shortcuts. Either they trust the agent's summary of its own work, or they skim a diff that is too large to read carefully. Both mean the bar for AI-written code ends up lower than the bar for human-written code. It should be the same, or higher.

The fix isn't to slow agents down. It's to put a gate between "the agent finished" and "this is a pull request," and to make that gate check evidence, not opinions.

What a quality gate should check

At AI CodeOps, every AI change waits at a release gate. It opens a pull request only when all of these hold:

1. The review actually ran. A review step reads the real diff and records structured findings. If the review couldn't finish, the change is held. A missing review never counts as a pass.
2. No serious findings are open. A high or critical finding that is still open, acknowledged or in progress blocks the change. To clear it, you fix it or triage it.
3. Your tests don't fail. The project's own test suite runs against the change in a sandbox. The agent may plan tests, but only a real test run counts.
4. The tested version is current. If someone edits the change after the tests ran, the old result no longer counts.
5. A person approves it. The person who started the session can't cast the approving vote while someone else in the workspace can. Editing the code after approval resets it to pending.
6. Workspace policy allows it. The repository and every changed file are checked against your policies before the pull request opens.

Fail closed, not open

The most important property of a gate is what it does when something goes wrong. If a check errors, the safe answer is "no." A gate that lets changes through when the reviewer times out or the test runner crashes isn't a gate. It's a suggestion.

The same goes for missing evidence. If the gate can't find a session for a change, it has nothing to judge, so it denies.

Gating merges outside the platform

A gate inside one tool only covers changes made through that tool. Someone can still merge directly on GitHub or GitLab. To cover that path, your CI can ask the gate for a verdict through an API, with a pull request URL or commit SHA, and fail the job when the answer is no. You can also make CI stricter per pipeline: block on medium findings, set a coverage floor, or require a change to be deployed to staging first.

Keep a record of how each change shipped

When something breaks in production, "the AI wrote it" isn't an answer. Each change should leave a trail: the task, the models used, the findings, the test runs, who approved it and how each person voted. In AI CodeOps that trail is a signed evidence pack that anyone can verify.

The bar should not move

AI agents are a real productivity gain. The way to keep that gain is to hold their work to the same standard as everyone else's: tests that ran on this exact version, a review that read the real diff, and a person who said yes.

AI CodeOps is free during the open beta and free forever for non-commercial open-source projects. Learn how the gate works at aicodeops.dev/product/quality-gates.

Originally published on Medium. Drafted with AI assistance and edited by the AI CodeOps team.