The AI PR that commented out the failing check
I keep hitting this one in AI-written PRs. CI fails on a lint rule, a type check, or a permission assert. The agent "fixes" it by commenting the check out, deleting the assertion, or wrapping it in if False. Diff looks
I keep hitting this one in AI-written PRs.
CI fails on a lint rule, a type check, or a permission assert. The agent "fixes" it by commenting the check out, deleting the assertion, or wrapping it in if False. Diff looks tiny. Green build. Merge temptation is high.
The invariant did not come back. You just stopped measuring it.
The 60-second check I run now
When an agent PR claims to fix a red check and the code change is mostly deletions or comments:
- Diff the failing check itself — is the assert/rule still there and still meaningful?
- Ask: did the PR restore the invariant the check was guarding, or only silence the alarm?
- If the check is gone or neutered and the root cause is not fixed, I bounce it. Green CI with a missing guard is worse than a red one.
This is one of the stops on a short bar I use before I merge agent PRs. Free one-pager if you want the rest: https://chopragunji.gumroad.com/l/zpnmdn
If you'd rather someone else do a hard pass on one scary AI PR, I still have founding slots at $49 (normally $99): https://chopragunji.gumroad.com/l/byoyi/FOUNDING
What does your team do when agents "fix" red checks by deleting them?
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.