Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 2 min read

Your AI agent shipped a billing bug. ProdVerdict blocks it in CI.

AI coding tools are fast. Tests pass. The PR looks fine. Then production quietly leaks revenue. The pattern I kept seeing in vibecoded SaaS: Billing logic lands in a PR Webhooks get skipped or half-wired Stripe says a

Your AI agent shipped a billing bug. ProdVerdict blocks it in CI.

AI coding tools are fast. Tests pass. The PR looks fine. Then production quietly leaks revenue.

The pattern I kept seeing in vibecoded SaaS:

  1. Billing logic lands in a PR
  2. Webhooks get skipped or half-wired
  3. Stripe says active while Postgres says has_paid_access = false

That's not a lint issue. That's money walking out the door.

ProdVerdict is deterministic CI for production contracts β€” not another AI code reviewer. Zero LLM in the evaluation path. Read-only observations (Stripe/Paddle + your DB) and fixed rules. Same input, same output every time.

Three contracts in v0.5

1. Access β€” billing vs database

Compares live subscription state (Stripe or Paddle) against your Postgres users table. Catches:

  • Revenue leak β€” active subscription, has_paid_access = false
  • Wrongful access β€” cancelled user still has paid features
  • Plan drift β€” price ID maps to pro, row says starter
  • Duplicate customer β€” same billing ID on multiple app users

2. Config β€” env var drift

Scans process.env references in your code vs .env.example and CI secrets. The classic "it worked locally" killer.

3. Migration β€” unsafe Postgres DDL

Static SQL rules for agent-generated migrations: blocking CREATE INDEX without CONCURRENTLY, dangerous NOT NULL on hot tables, and more.

Try it in 10 seconds β€” no API keys

Clone the SDK once for fixture paths, then run:

git clone --depth=1 https://github.com/prodv-dev/prodverdict-sdk.git
cd prodverdict-sdk

npx prodverdict check access \
  --config examples/paddle-stripe/prodverdict.yml \
  --fixtures \
  --fixtures-dir examples/paddle-stripe/scenarios/fail-revenue-leak

You should see a FAIL with a concrete fix hint:


[HIGH] user:usr_alice β€” Active subscription but has_paid_access is false
fix: Set has_paid_access=true in your webhook handler
VERDICT: FAIL

Passing scenario:

npx prodverdict check access \
  --config examples/nextjs-stripe/prodverdict.yml \
  --fixtures \
  --fixtures-dir examples/nextjs-stripe/scenarios/pass

Add to your repo

npx prodverdict init --stack nextjs-stripe
# or: npx prodverdict init --stack paddle-stripe

Commit prodverdict.yml, add secrets, and wire the GitHub Action:

- uses: prodv-dev/[email protected]
  with:
    config: ./prodverdict.yml
    contract: access
  env:
    STRIPE_SECRET_KEY: ${{ secrets.STRIPE_SECRET_KEY }}
    DATABASE_URL: ${{ secrets.DATABASE_URL }}
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Runs on every PR. Secrets never leave your runner. Optional dashboard stores verdict metadata only β€” not subscription rows or PII.

Install from GitHub Marketplace Β· npm Β· MCP for Cursor/Claude

How this differs from AI code review

AI code review ProdVerdict
Evaluation LLM suggestions Deterministic rules
Input Source code diff Live billing + DB state (or fixtures)
Output "Consider checking…" PASS / WARN / FAIL with fix hints
CI behavior Optional, noisy Merge-blocking on HIGH

Agents can call ProdVerdict via MCP before opening a PR. Enforcement still happens in CI.

Pricing

  • Free β€” public repos, CLI, GitHub Action, MCP, Access + Config contracts
  • Pro ($39/project/mo) β€” private repos, scheduled checks, dashboard history

We're on Product Hunt β€” use code PRODUCTHUNT at checkout for 3 months free Pro.

What's next?

Webhook contract, API boundary checks (mass-assignment / forbidden fields), and more stack templates. Open an issue on prodverdict-sdk if you want a rule prioritized.

What's the scariest prod bug an agent ever shipped for you? Drop it in the comments β€” I'm collecting patterns for the next contracts.

ProdVerdict is MIT-licensed (SDK + Action). prodverdict.com Β· Discord

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.