We applied the audit's fixes to the broken demo app and scanned it again. What went green, what stayed red, and what we got wrong

Disclosure: Grey State Development built LaunchVet, the tool used here. The free core is MIT licensed. Everything below can be repeated by hand, and the demo and the fixes are in the repo.

In the first article we audited a deliberately broken resume app (ResumeFlash, Next.js, Supabase, Stripe) and listed ten of the 28 findings with their fix prompts. Reasonable question: do the prompts actually fix anything, or do they just produce a better-looking report? So we took a copy of the demo, applied the fixes, and scanned it again.

The setup, so you can judge the result. The fixes were written by Claude, following the instruction text in each finding's fix prompt, in a copy of the demo app: 32 files changed or added. Then we re-ran the LaunchVet scanner (v0.1.3) on the copy and read the code again with the judgment checks in mind. We did not build or run the app. We did not run any of the verify steps the prompts ask for (the Stripe CLI test, the two-user RLS test, the curl checks). We did not run the SQL on a real Supabase project. So this is a before and after of what the scanner can see, plus a code review, not proof that the app works.

The scanner, before and after

The scanner is the pattern-based half of the audit: 16 rules. It reads files and does not understand intent. On the broken app it returned 21 findings (after grouping repeated hits): 5 critical, 9 high, 6 medium, 1 low. After the fixes the first scan, with v0.1.2, still returned 3: 1 critical, 1 high, 1 medium. Those three were a bug in the scanner, explained below. It is fixed in v0.1.3, and with it the fixed copy scans with 0 findings.

LV006 is the row that did not clear on the first scan, and the reason is worth knowing.

The three that stayed on the first scan: a fix the scanner could not see

The fix prompt for the missing Row Level Security says to leave the old migration alone and add a new one that turns RLS on and replaces the "allow everyone" policies. That is the right practice: you do not edit a migration that has already run. We did exactly that, in a new file, 20261102000000_fix_rls.sql.

The scanner then still reported the old file, because it read each migration on its own and did not know a later one fixed it. Those three findings, in 20261101000000_init.sql, were false alarms against the final database. That was a bug in LaunchVet, not in the fix. In v0.1.3 the scanner follows later migrations, so a migration that enables RLS or replaces a policy clears the earlier finding, and the fixed copy now scans with 0 findings.

A clean scan means the scanner found nothing it knows how to look for. It is not a sign the app is safe, as the next two sections show.

What we got wrong

Applying fixes is where a second set of bugs shows up. In our first pass:

1. The Stripe webhook saved the customer with an upsert on user_id, but the table had no unique constraint on that column, so the upsert would not have worked. We only caught it by reading the code again. The fix was a one-line unique constraint in the migration and an onConflict option.
2. The first lockfile regeneration failed. @supabase/ssr needed a newer @supabase/supabase-js than the demo pinned, so the install stopped with a peer dependency error. We bumped the version and regenerated it. The scanner's lockfile finding went away only after that.
3. The webhook records each event id before handling it. If the handler then fails, Stripe retries, the retry is seen as a duplicate, and the upgrade is lost. A safer version records the id only after the work succeeds, or wraps both in a transaction. We left it as written to show you the gap.
4. The URL import checks the host with a DNS lookup and then calls fetch, which looks the host up again. A host that answers differently the second time can pass the check and still reach an internal address. Real SSRF protection needs the connection pinned to the address you checked.
5. The rate limiter reads the client address from x-forwarded-for. That header is only trustworthy if your host overwrites it. Check how yours behaves.

None of these are visible to a regex. They are the reason the report has a "Needs a manual check" list, and the reason we would not call the fixed copy safe.

What still needs a human

• Create the private uploads bucket in Supabase and test that other users cannot read it.
• Create the Stripe price, set STRIPE_PRICE_PRO and the webhook secret, turn on the customer portal and trial-ending emails.
• Set a monthly spend limit at your AI provider and create the Upstash Redis database the limiter uses.
• Replace the placeholder support address, and have a lawyer read the privacy policy, terms and refund pages. They are starter templates written for a demo, not legal advice.
• Run the verify steps from each prompt. We did not.

What we would take from this

The fix prompts did their job on the mechanical problems: keys out of the browser, ownership checks, signed webhooks, parameterized SQL, pages that were missing. A scanner re-run is a cheap way to see that you did not forget one. The cost is that the prompts, like any AI-written code, create new gaps, so read the diff, run the verify step, and scan again.

If you want to repeat this on your own app, install it in Claude Code:

/plugin marketplace add Lt23live/launchvet
/plugin install launchvet@launchvet

Then run /launchvet:launchvet, or ask "Run LaunchVet on this app." You need Node.js 18 or newer for the scanner. It never edits your code, and it writes one file, launchvet-report.md. LaunchVet runs inside your own Claude Code session. There is no LaunchVet server, no signup, and nothing is uploaded to us. The scanner script makes no network calls. Claude Code itself still reads your files and talks to Anthropic the way it does for any task.

The free core has 26 checks. Pro is a one-time $19 on Gumroad ($12 until 11 Nov 2026, 11:59 pm PT) and adds more checks, payments and Stripe readiness, an app store module, legal page starter templates and a CLAUDE.md security rulebook. You do not need it to fix what the free core finds.

Five common launch blockers with a by-hand check for each, no tool needed: https://greystatedevelopment.com/vibe-coded-app-launch-blockers/
Project page: https://greystatedevelopment.com/launchvet/
Repo (free core, install): https://github.com/Lt23live/launchvet
First article's full demo report: https://github.com/Lt23live/launchvet/blob/main/examples/broken-app/launchvet-report.md
Pro on Gumroad: https://greystatedevelopment.gumroad.com/l/launchvet/LAUNCH12

LaunchVet is an automated checklist, not a security guarantee or legal advice. AI is used in this product and can make mistakes.

What did the last set of AI-written fixes in your project get wrong?