FAQ: A Sandbox Build Is Not a Release Receipt

Did the agent compile your app on a scratch host?
Did you almost treat that log as a release?
I keep hearing both claims in pull requests.

This FAQ kills five claims I still hear.
Each claim dies with a command you already have.
No dashboard screenshot counts as evidence here.

What this FAQ covers

I am talking about code that only built remotely.
I am not talking about model leaderboard drama.
I am not replacing your real pipeline either.

A scratch box is a compiler for ideas.
A release is a hashed tree plus recorded commands.
Those two objects are easy to confuse today.

The loop I actually trust

I still let an agent thrash in a sandbox.
Sometimes that sandbox is a free remote server.
MonkeyCode is one place I use for that loop.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

It has free model access and a free server option.
I still refuse to ship whatever that box compiled.
The receipt workflow below works on any host.

Myth 1: A sandbox compile is a release candidate

Claim: If it built there, it can ship.

What people repeat: The compiler already blessed the tree.
What that log proves: Some compiler accepted some files.
What it does not prove: Your release toolchain accepts them.

Ask one rude question before you tag anything.
Which exact compiler binary produced that success line?
If you cannot name it, you do not have a receipt.

# On the sandbox, after the agent shouts success:
which cc || true
cc --version || true
npm -v && node -v || true
python -V && pip -V || true
go env GOVERSION GOTOOLCHAIN || true
git rev-parse HEAD
git status --porcelain=v1

Copy those lines into the pull request body.
If the agent cannot produce them, stop merging.
A green sentence in chat is not a compiler identity.

Corrected mental model

Treat the sandbox as a noisy preview compiler.
Treat your runner as the only compiler that counts.
Preview compilers do not get to mint versions.

Myth 2: The sandbox toolchain matches production

Claim: It is Linux, so the versions are close enough.

What people repeat: The container felt normal enough.
What you can measure: File names under /usr lie often.
What you still need: A pinned toolchain file you own.

# Record what the sandbox actually is.
uname -a
cat /etc/os-release
command -v node && node -p process.version
command -v python3 && python3 -c 'import sys; print(sys.version)'
command -v rustc && rustc -V

Now run the same block on your release image.
Diff the two text files without mercy or hope.
A missing minor version still breaks native addons.

I keep a tiny allowlist, not a vibe check.
If node drifts, the receipt must fail closed.
Close is boring. Boring is the point here.

# toolchain.allowlist example
node==20.17.0
python==3.12.6

Corrected mental model

OS family is not a toolchain pin.
Pins live in files you review and hash.
Everything else is tourism, not packaging.

Myth 3: The agent saying "build succeeded" is an artifact

Claim: The model reported success, so the process did.

What people repeat: The last message looked official.
What that string is: Generated prose about a command.
What an artifact is: Bytes plus an exit code you stored.

Would you accept a human saying "tests passed"?
Would you skip JUnit because the standup sounded confident?
Then do not accept a chat bubble as exit 0.

# Wrap the real command. Ignore the model's summary.
set -o pipefail
./scripts/build.sh > /tmp/build.stdout 2> /tmp/build.stderr
echo $? > /tmp/build.exit
wc -l /tmp/build.stdout /tmp/build.stderr
sha256sum /tmp/build.stdout /tmp/build.stderr /tmp/build.exit

Check the exit file, not the assistant paragraph.
If the wrapper never ran, you have theater.
Theater does not belong in a release checklist.

Corrected mental model

Summaries are commentary. Receipts are files.
Commentary can be wrong and still sound calm.
Files can be hashed. Calm cannot be hashed.

Myth 4: The UI file list is the working tree

Claim: The sidebar showed the patch, so disk matches.

What people repeat: I watched the diff render live.
What a UI list is: A view the tool chose to paint.
What git cares about: Blobs in the index and worktree.

I do not argue with screenshots anymore.
I ask for names, sizes, and hashes instead.
If those three disagree, the UI lost.

git diff --name-only
git diff --stat
git ls-files -v | awk '$1 ~ /[S?]/ {print}'
find . -type f -not -path './.git/*' -print0 | \
  sort -z | xargs -0 sha256sum > /tmp/tree.sha256

Then compare /tmp/tree.sha256 with your laptop copy.
One missing file ends the debate immediately.
Pretty file trees are not Merkle trees.

Corrected mental model

The working tree is the product.
The chat panel is a projector.
Projectors do not ship.

Myth 5: The same prompt rebuilds the same tree tomorrow

Claim: I can recreate this with the original request.

What people repeat: The prompt is the build script.
What a prompt is: A wish with unbounded inputs.
What a rebuild needs: Bytes, commands, and versions frozen.

Prompts do not pin package indexes.
Prompts do not pin compiler patches.
Prompts do not pin the agent's extra tool calls.

If you need the tree again, store the tree.
Store the commands that mutated it too.
Do not store a vibe and call it provenance.

git add -A
git diff --cached > /tmp/agent.patch
sha256sum /tmp/agent.patch
# Keep the patch. Throw away the prompt fan fiction.

Corrected mental model

Reproducibility starts after the agent stops talking.
Your job is to freeze the result, not the chat.
Chat is grease. Grease is not a spec.

Artifact: a local release receipt

I use one script after every sandbox session.
It does not care which host wrote the files.
It fails if you only have a story.

Save this as sandbox_receipt.py and run it locally.

#!/usr/bin/env python3
"""Record a sandbox session as a hashed receipt.

Label: example script. Run it on a git checkout you control.
It does not talk to any vendor API.
"""
from __future__ import annotations

import argparse
import hashlib
import json
import os
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path


def run(cmd: list[str]) -> tuple[int, str]:
    proc = subprocess.run(cmd, text=True, capture_output=True, check=False)
    out = (proc.stdout or "") + (proc.stderr or "")
    return proc.returncode, out.strip()


def sha256_file(path: Path) -> str:
    h = hashlib.sha256()
    with path.open("rb") as fh:
        for chunk in iter(lambda: fh.read(65536), b""):
            h.update(chunk)
    return h.hexdigest()


def toolchain() -> dict[str, str]:
    keys = {
        "uname": ["uname", "-a"],
        "node": ["node", "-v"],
        "python": ["python3", "-V"],
        "git": ["git", "--version"],
    }
    found: dict[str, str] = {}
    for name, cmd in keys.items():
        code, text = run(cmd)
        found[name] = text if code == 0 else f"missing:{code}"
    return found


def changed_files() -> list[str]:
    code, text = run(["git", "diff", "--name-only", "HEAD"])
    if code != 0:
        raise SystemExit("git diff failed; are you in a repo?")
    extra_code, extra = run(["git", "ls-files", "--others", "--exclude-standard"])
    if extra_code != 0:
        raise SystemExit("git ls-files failed")
    names = [line for line in (text.splitlines() + extra.splitlines()) if line]
    return sorted(set(names))


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--build-cmd", default="")
    parser.add_argument("--out", default="receipt.json")
    args = parser.parse_args()

    head_code, head = run(["git", "rev-parse", "HEAD"])
    status_code, status = run(["git", "status", "--porcelain=v1"])
    if head_code != 0 or status_code != 0:
        print("Need a git checkout you control.", file=sys.stderr)
        return 2

    files = changed_files()
    hashed = []
    missing = []
    for name in files:
        path = Path(name)
        if not path.is_file():
            missing.append(name)
            continue
        hashed.append({"path": name, "sha256": sha256_file(path), "bytes": path.stat().st_size})

    build = {"skipped": True, "exit": None, "tail": ""}
    if args.build_cmd:
        code, text = run(["bash", "-lc", args.build_cmd])
        build = {"skipped": False, "exit": code, "tail": text[-2000:]}

    receipt = {
        "recorded_at": datetime.now(timezone.utc).isoformat(),
        "cwd": os.getcwd(),
        "git_head": head,
        "git_status": status.splitlines(),
        "toolchain": toolchain(),
        "changed_file_count": len(files),
        "changed_files": hashed,
        "missing_paths": missing,
        "build": build,
    }
    Path(args.out).write_text(json.dumps(receipt, indent=2) + "\n")
    print(f"wrote {args.out} with {len(hashed)} hashed files")
    if missing:
        print("missing paths:", ", ".join(missing), file=sys.stderr)
        return 3
    if not hashed:
        print("no changed files; nothing to receipt", file=sys.stderr)
        return 4
    if args.build_cmd and build["exit"] != 0:
        return 5
    return 0


if __name__ == "__main__":
    sys.exit(main())

Run it on the machine you actually own.
Pull the patch first if the sandbox still holds it.
Then force a local rebuild you can name.

python3 sandbox_receipt.py --build-cmd 'npm test --silent' --out receipt.json
python3 -c 'import json; print(json.load(open("receipt.json"))["build"])'

Attach receipt.json to the pull request.
Argue with the JSON, not the chat transcript.
If the JSON is thin, the change is not ready.

Decision table

Print that table next to the PR template.
It stops a surprising number of fake merges.
People hate it, then they start using it.

Limitations

This receipt is not supply-chain provenance.
It is not SLSA, not Sigstore, not a bill of materials.
It will not catch a malicious registry swap.

It also will not freeze a vanished free host.
Do not keep secrets on a scratch server.
Do not treat a deleted box as an audit log.

Hashes of source are not hashes of images.
A local npm test is not production traffic.
If you need bit-for-bit builds, use a real hermetic system.

Who should not use this

Skip this if you already have hermetic CI.
You do not need another JSON souvenir then.
Skip this if you cannot run git locally.

Skip this for regulated binaries with named signers.
A receipt script is not a signing ceremony.
Skip this if the sandbox still holds production secrets.

What I want in review

I want three files, not a victory speech.
I want the patch, the receipt, and the command log.
I want the toolchain block to match the release image.

Can the agent still help? Yes, in the sandbox.
Can the sandbox mint a release? No, never.
That split is the whole FAQ.

Steal the script. Then argue with the JSON.