pytest for prompts: a zero-dependency CLI diff to catch regressions before they ship

You edit a system prompt or switch models, and other cases quietly break. prompt-diff runs a small prompt suite against a baseline (A) and a candidate (B). It writes a side-by-side line diff and a pass/fail report in Markdown and JSON. If B fails a case that A passed, it exits with code 1, so you can gate CI on it.

What you get:

• prompt-diff.mjs: a single-file Node 18+ CLI with zero dependencies
• Checks: contains, notContains, regex, minChars/maxChars, valid JSON
• Two variant sources: recorded outputs (.jsonl, offline) or any OpenAI-compatible endpoint
• Statuses: PASS, FAIL, REGRESSION, FIXED
• A working example suite with A/B outputs
• A README with a GitHub Actions snippet

Pay what you want: minimum $1, suggested $5. MIT license.

Get it here: https://renevibe76.gumroad.com/l/tywvdr