I scanned 25 vibe-coded apps. The API keys were fine. The server wasn't.

Everyone's favorite vibe-coding horror story is the API key pasted into the frontend. So I went looking for it.

I took 25 public vibe-coded apps on GitHub, scanned every file and every line of their git history, and then read their server code by hand.

The keys were fine. Not one leaked. What I found instead was a hidden key behind a door anyone could walk through.

How I did it

Finding the repos. I used GitHub's search API with queries like "lovable.dev/projects" in:readme stars:>=3, plus searches for Bolt and v0 projects. I kept repos that were active, had more than a template's worth of code, and clearly came from a builder tool: 19 Lovable, 3 Bolt, 2 v0. (Cursor and Replit don't leave a reliable public fingerprint, so none made this batch.)

Scanner pass. I ran secret, Supabase, Firebase and cost-trap detectors over every text file at HEAD (2,269 files, skipping node_modules, lockfiles, build output and binaries), then over every added line in the full history (git log -p --all).

Manual pass. I read the server code myself: supabase/functions/*, supabase/config.toml, SQL migrations, Next.js app/api/* routes, middleware.ts, and Express servers.

What I didn't do. I didn't contact any deployed app or call any endpoint, and I didn't copy any secret values. Everything here comes from public source code. I'm not naming any repo, because some of these endpoints are probably still live.

What went right

This part deserves credit.

• 0 of 25 repos had a hardcoded secret (OpenAI, Anthropic, Stripe secret, AWS, GitHub, Resend, private keys or generic). The same was true across full git history.
• 0 of 25 had the Supabase service_role key in code.
• 43 of 43 Supabase tables defined in migrations (across 4 repos) had Row Level Security enabled.
• 4 repos had a committed .env file, but 3 held only the Supabase URL, project id and publishable key, and the fourth had placeholder or localhost values. 0 live secrets.

I also saw a Stripe webhook that properly verified signatures with constructEvent, and an Express API with auth and rate-limit middleware.

So "don't put your key in the frontend" has sunk in, both for builders and for the AI tools writing the code. The next lesson hasn't.

The patterns

Context first: 11 of the 25 repos had any server or data layer at all. The other 14 were frontend-only. So the numbers below are small, and I'll say exactly how small.

1. The key is hidden, but the endpoint is open

Two repos called an AI model from the server. Both kept the key in server env vars, which is right. Both exposed the route that uses it with no login check and no per-user rate limit. One was a pair of Supabase edge functions with verify_jwt = false and CORS *. The other was three Next.js API routes with no middleware.

That's n=2, not a statistic. But the result is the same either way: anyone who finds the URL can spend your credits.

The bad pattern looks something like this (a generic example, not from any repo):

// app/api/generate/route.ts
export async function POST(req: Request) {
  const { prompt } = await req.json();
  const res = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: prompt }],
  });
  return Response.json(res.choices[0].message);
}

The fix is to require a user and limit them:

export async function POST(req: Request) {
  const user = await getSessionUser(req);            // your auth
  if (!user) return new Response("Unauthorized", { status: 401 });

  const { success } = await ratelimit.limit(user.id); // e.g. Upstash, or a DB counter
  if (!success) return new Response("Too many requests", { status: 429 });

  const { prompt } = await req.json();
  // ...same model call
}

Then set a hard monthly spend cap in your AI provider's dashboard. That's your last line of defense if the first two fail.

2. Auth switched off right next to the admin key

One push-notification edge function had this in supabase/config.toml:

[functions.send-notification]
verify_jwt = false

Inside, it built a Supabase client with the service_role key, which bypasses RLS, and sent notifications whose title, body and recipients came straight from the request body. If no recipients were given, it sent to every subscriber. So anyone could push official-looking alerts to every user.

Here's a generic version of the shape:

const admin = createClient(url, Deno.env.get("SUPABASE_SERVICE_ROLE_KEY")!);
const { title, body, userIds } = await req.json();
const targets = userIds ?? (await admin.from("subscriptions").select("*")).data;
// ...send to targets

The fix: leave verify_jwt on (it's the default), check who's calling before you touch the admin client, and never let the request choose "everyone."

const authed = createClient(url, anonKey, {
  global: { headers: { Authorization: req.headers.get("Authorization")! } },
});
const { data: { user } } = await authed.auth.getUser();
if (!user || !(await isAdmin(user.id))) {
  return new Response("Forbidden", { status: 403 });
}
if (!Array.isArray(userIds) || userIds.length === 0) {
  return new Response("Recipients required", { status: 400 });
}

3. The policy name lies

One repo had an RLS policy named "Allow insert for authenticated users." It looked something like this:

create policy "Allow insert for authenticated users"
  on notifications for insert
  with check (true);

There's no to authenticated, so it applies to every role, logged in or not. And with check (true) means any caller can insert a row for any user. RLS was technically on, but this policy let anyone through.

create policy "Users insert their own notifications"
  on notifications for insert
  to authenticated
  with check (auth.uid() = user_id);

Or don't allow client inserts at all, and create these rows from a trigger or a trusted server function.

4. Committed .env files

4 of 25 repos committed a .env. None held a live secret this time, but that's luck, not a process. Add .env* to .gitignore, commit a .env.example with placeholder values, and if a real secret ever lands in history, rotate it. Deleting the file doesn't remove it from git history.

Check your own app in 10 minutes

You can do all of this by hand:

1. Grep your server code for AI calls. For each route that calls a model, ask: what stops a logged-out stranger from calling this in a loop?
2. Open supabase/config.toml. Any verify_jwt = false? Each one needs a written reason (a webhook that verifies its own signature is a good one).
3. Search functions for SERVICE_ROLE. Anywhere it shows up, the caller should be checked first.
4. Search migrations for using (true) and with check (true). Also check every policy for a to clause.
5. Look for request bodies that pick targets. User ids, recipient lists, "all."
6. Run git log --all -p | grep -iE "sk-|secret|service_role". Rotate anything real you find.
7. Set a spend cap with every AI provider you use.

Caveats

• Small numbers. 2 of 2 is n=2. I'm saying "both," not "100%."
• Sample bias. The repos came from GitHub search, filtered by stars and heavy on Lovable. This is a snapshot, not a survey, and a public repo may not match what's deployed.
• Manual findings. Patterns 1 to 3 came from reading code by hand, not from the automated detectors.
• One false positive excluded. A voice-notes app was flagged for an AI key in the client, but it was the user's own bring-your-own key stored locally, so I left it out.
• Public keys aren't leaks. 5 repos shipped the Supabase anon key and URL, and 1 shipped a Firebase web config. Those are public by design, so I didn't count them. They do mean your RLS has to be right.

The tool

I turned these checks into HackOrBroke. Paste your app's URL, and optionally a public GitHub repo. It looks for leaked keys in your JS bundles and exposed files like /.env and /.git/config. It also runs static analysis on the repo for the server patterns above: open AI routes, verify_jwt = false with the service role key, and loose RLS policies. It never calls your app's APIs.

The scan is free. The full report is $19 and adds file paths, line numbers and a fix prompt you can paste into your AI tool. Fair warning: auth detection is heuristic, it only reads public repos, and it doesn't support Python yet.

(Small aside: the business is run day to day by an AI agent I call Frankie, and it was built and launched in a day.)

If it flags something that isn't a problem, please tell me. False positive reports are the feedback I need most right now.