Vibe Coding Got My AI Product Live. Production Engineering Showed Me the Gaps.

Building with AI made Cernalo move fast: architecture, auth, payments, desktop packaging, cloud, even a Windows app on the Microsoft Store.

Going live was the easy part. Keeping it reliable was not.

Here are four gaps AI-generated code didn't catch for me.

1. Green CI ≠ a working product

Hundreds of tests can pass while real customer journeys fail. Microphone use, system tray, window management, and actual Quit on Windows needed validation beyond unit tests. Test the full path a customer takes.

2. The environment can fail when the code is fine

I hit spawn EPERM and almost "fixed" packaging. The restricted test environment was blocking Node child processes. FFmpeg worked when run directly. Ask first: code, dependency, OS, or test environment?

3. The build you tested may not be the build you ship

One AppX package was missing latest compiled changes. Repo current, tests green, artifact old. Rebuild and verify the release candidate against real workflows. A successful build is not a successful release.

4. A healthy API isn't a healthy business

HTTP 200 doesn't prove subscriptions, webhooks, entitlements, and auth all work. Paid products need observability for the business path, not only the service path.

What changed for me

I'm moving from feature-first to evidence-first: freeze release candidates, track exact artifacts, validate real user flows, and treat AI fixes as hypotheses until proven.

Vibe coding gets you from idea to working product. Production engineering gets you from working to dependable.

If you're building with AI, what's been your hardest jump from prototype to production?