ProofPack: Automating the Click-Through After the Feature Is Built

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Finishing a feature often leaves another chore: click through it again, check what happened, take screenshots, and assemble an explanation someone else can follow.

I built ProofPack to handle that follow-up work.

ProofPack is an open-source, local-first application that turns a feature brief into a bounded browser demonstration with executable checks and saved evidence. It helps developers demonstrate observable behavior without manually collecting every screenshot and reconstructing every step.

The user supplies an allowlisted application URL, a description of the behavior, and expected outcomes. For example:

Create an account and a project. Log out, open a fresh browser session, and log back in. Verify that the project is still there.

An AI model creates a structured plan and chooses actions from observations of the page. A separate browser worker performs those actions, executes assertions, and records what happened.

The user receives an assertion ledger, an execution timeline, screenshots, downloadable browser traces, and Markdown or self-contained HTML reports.

The distinction I wanted to preserve is simple: asking a model what should happen is different from collecting evidence of what did happen. In ProofPack, the model proposes the work. Executable checks determine the outcome.

Demo

Watch the ProofPack demonstration on YouTube

The video shows ProofPack running locally with real Groq inference against Wikipedia. The request is to search for Go (programming language) using Wikipedia’s search field, then verify the destination URL and visible article title.

Both assertions passed in the final attempt.

Watch the connection between the request, recorded actions, assertion results, and exported evidence. The standalone HTML report includes its screenshots and opens directly from disk.

The video uses selected recording segments with waiting omitted. The article is also opened separately for review after the AI run. An earlier screenshot timeout remains in the run history; the successful retry does not erase it.

I also built two controlled target applications called LaunchDesk. They support the same account-and-project workflow, but one deliberately loses project visibility after a fresh login.

Recorded live Groq runs show the contrast: the working version passed 4/4 checks, while the broken version passed 3/4, failing the final persistence check. Screenshots preserve the observed missing project.

LaunchDesk is a controlled demonstration of the workflow. The Wikipedia example separately exercises a read-only interaction with a public website.

ProofPack itself runs locally; there is no public hosted dashboard.

Code

Source code: github.com/Rohan-Saxena644/ProofPack

The repository contains the application, separate worker, controlled targets, model adapters, verification scripts, screenshots, and recorded verification results.

To try it, install dependencies with npm ci, install Chromium with npm run browsers, copy .env.example to .env, and run npm run dev:all.

The default fixture mode needs no model key. It uses deterministic decisions with a real browser and is explicitly labeled. Live mode requires provider configuration and credentials.

The original application source is MIT-licensed. Dependencies and model weights have their own licenses.

How I Built It

The demonstrated AI configuration uses OpenAI’s open-weight GPT-OSS 120B model, identified as openai/gpt-oss-120b, through Groq’s hosted inference service. Groq serves the model; inference is not running on my laptop.

The execution path starts in a Next.js and React dashboard. The API validates the request and queues it in SQLite. A separate Node.js worker claims the job and runs Playwright Chromium, keeping long browser sessions outside the web request lifecycle.

The planner produces named goals and explicit assertion specifications. Zod validates that structure, including the requirement that every assertion belongs to exactly one goal.

During execution, the worker observes bounded page text and accessible controls. The model receives control IDs, labels, roles, relevant form state, and previously executed actions. It chooses from a small action vocabulary: fill, click, select, navigate, assert, capture, start a fresh browser context, or complete a goal.

The model cannot submit arbitrary JavaScript, selectors, shell commands, or Playwright code. The executor owns those operations.

Assertions currently check exact visible text or exact URLs. Their expected values, observed results, and evidence references are persisted. Finalization checks the ledger before accepting a pass, and report generation checks it again. Reports are assembled deterministically, so a summarizing model cannot rewrite a failed result into a success.

One implementation problem came from the meaning of past actions. Control IDs belong to individual page observations. Earlier live runs repeated work when the model interpreted old IDs against the current page. I added execution provenance, before-and-after URLs, current-goal context, and guards against unchanged repeated actions.

The Wikipedia run exposed another practical limit: long observations could exceed the provider’s token allowance, and full-page screenshots could time out. I compacted model-facing control metadata, bounded the text sent for decisions, and switched evidence capture to viewport screenshots. Text assertions scroll their target into view before capture.

Failure handling also matters. A contradicted assertion is Failed. An unavailable target, exhausted budget, or model problem is Blocked. Worker leases expose interrupted attempts; a manual retry starts fresh browser state and retains earlier evidence.

Shared quota scheduling coordinates workers using the same database, but cannot create additional provider capacity. Exact-origin restrictions constrain browser traffic. Generated passwords are substituted by the executor, masked in screenshots, and redacted from finalized traces.

Verification includes 28 automated unit and improvement tests, eight fixture browser verification groups, and seven additional production UI groups. Fixture decisions are separate from real inference. Recorded Groq runs independently exercised working and broken persistence, and the latest public-site run passed both Wikipedia checks. Type checking, lint, and the production build also passed.

Why Does Open Innovation Matter?

For ProofPack, openness makes the automation inspectable and adaptable.

The source exposes how plans are validated, which actions are permitted, how assertions work, and how evidence becomes a report. Someone can change those rules instead of depending on an opaque demonstration service.

The adapter boundary also separates model choice from browser execution. Groq with GPT-OSS 120B is the configuration I actually verified. OpenRouter and local OpenAI-compatible endpoints are implemented, but real inference through those paths remains unverified.

The current demonstration is therefore not offline. Briefs and bounded page observations are sent to Groq. Screenshots and traces remain local.

I also keep the application’s MIT license separate from model licensing. Open components offer useful control and replacement options; they do not remove the need to validate outputs or check behavior.

The Lazy Payoff

During the build, ProofPack performed the repeated account creation, project creation, logout, fresh-session login, and persistence checks against LaunchDesk. It also performed the public Wikipedia search and collected its evidence.

That replaces a concrete sequence of manual browser interactions and evidence assembly. I did not measure a comparable manual baseline, so I am not claiming a particular number of minutes saved.

The user still defines success, configures allowed targets, reviews the proposed coverage and results, and investigates failures. ProofPack does not establish that an application is bug-free or support every website workflow.

The payoff is having the exercised steps and their evidence ready to inspect, rather than having to repeat the walkthrough and piece the explanation together afterward.