Label the Miss on a Four-Cell Replay Before Anyone Requests a Freeze

An agent-patch miss stays unlabeled until a four-cell replay places it. Two cells run the base revision. Two run the candidate patch. Each revision is paired once with the locked fixture and once with the fixture the patch proposes.

A red patch cell beside a green base cell is a patch defect. That row is not a flake. It is not a freeze candidate either.

A single failing check cannot carry the distinction. The same red badge covers a broken property, a moved fixture, an interaction of the two, and noise that also fails on base. Those outcomes do not share a next step. Folding them into one freeze request teaches the gate to hide patch bugs inside an inconclusive bucket.

The procedure below sits in front of whatever freeze rule you already run. It does not add a ledger. It does not grant admission. It forces a label, computed from pins a reviewer can recompute, before the word flake enters the discussion.

Why one red check is under-specified

Agent patches often edit production code and test inputs in the same diff. A pricing change arrives with a new order fixture. A retry change arrives with a new timeout table. The suite goes red.

The author, human or model, then points at history. This property has failed before, so freeze it. That inference skips the axis that moved.

If the new fixture fails on the old code, the fixture drifted. If the old fixture fails on the new code, the patch broke a predicate the suite already trusted. If neither axis fails alone and the combination fails, the interaction is still what you propose to ship. None of those three is noise.

Noise is the residual case. It requires a mostly green control cell, plus the same miss on a revision that is not only the patch. Even then the label is a candidate for an inconclusive path you already operate. It is not a pass.

Pin both axes before any cell runs

Keep the names boring so a log line stays greppable. code is base or patch. fixture is locked or candidate. The control cell is (base, locked).

If that cell is red, the other three are not interpretable. Emit CONTROL_BROKEN and stop. Do not average a red control with a green neighbor and call the mean acceptable.

The remaining cells each ask one question.

1. (patch, locked) asks whether the patch breaks a property while the fixture stays still.
2. (base, candidate) asks whether the candidate fixture breaks code you already shipped.
3. (patch, candidate) asks what the combination does. It must not outvote a single-axis failure.

A property check is a predicate over inputs from a declared generator, seed, and trial count. A fixture lock is a content digest plus a schema version. A directory path is not a lock. Change the seed, the trial count, or the generator version between cells, and the comparison is void.

Numbered triage

Fail closed. A missing pin is a failed run, not an optional field.

1. Record base_rev, patch_rev, fixture_locked_digest, fixture_candidate_digest, generator_version, seed, and trials. Refuse to start when any value is empty. Also refuse when the two fixture digests match and the patch still claims a fixture edit.
2. Run (base, locked) on every selected property. A red property yields CONTROL_BROKEN for that property. Do not open a freeze request from this state.
3. Run (patch, locked) with the same seed and the same trial count. Red here, with a green control, is PATCH_DEFECT.
4. Run (base, candidate) under those same pins. Red here, with a green control, is FIXTURE_DRIFT.
5. Run (patch, candidate) only after both single-axis results are stored. Use it to mark INTERACTION or to confirm BOTH_AXES. A green combination cell does not erase a red single-axis cell.
6. Repeat the four cells a declared number of times. Three repeats are a reproduction check, not a confidence interval. A miss that never returns is an unstable signal. Record that fact, and do not promote it.
7. Write the cell map into the review. A freeze request with no map is incomplete. A map is still not permission to record the freeze.

Identical fixture digests are fine when the patch does not touch fixtures. Cells 4 and 5 then duplicate cells 1 and 3. Keep the duplicates in the record. An explicit copy is easier to audit than a hole.

Decision table

Read each row from left to right. The freeze column is almost entirely no. That is the point of the table.

BOTH_AXES means two defects arrived in one diff. Split them. INTERACTION means the patch is still responsible for the combination you want to merge. Calling either row a flake deletes the signal the next reviewer needed.

CLEAN needs no freeze. If a request shows up anyway, the map is being used as a bypass. Reject it.

A pure classifier and a record shape

The module is a proposal. It is not a measured run, and it does not call a model. Point the cell runner at your own harness. Keep label_property pure so anyone can recompute the verdict from stored booleans.

from enum import Enum

class Label(Enum):
    CONTROL_BROKEN = "control_broken"
    PATCH_DEFECT = "patch_defect"
    FIXTURE_DRIFT = "fixture_drift"
    BOTH_AXES = "both_axes"
    INTERACTION = "interaction"
    CLEAN = "clean"
    NOISE_CANDIDATE = "noise_candidate"

def label_property(control, patch_locked, base_candidate, patch_candidate, base_also_missed):
    if not control:
        return Label.CONTROL_BROKEN
    patch_bad = not patch_locked
    fixture_bad = not base_candidate
    if patch_bad and fixture_bad:
        return Label.BOTH_AXES
    if patch_bad:
        return Label.PATCH_DEFECT
    if fixture_bad:
        return Label.FIXTURE_DRIFT
    if not patch_candidate:
        return Label.INTERACTION
    if base_also_missed:
        return Label.NOISE_CANDIDATE
    return Label.CLEAN

def freeze_request_allowed(label: Label) -> bool:
    # Filing a request is not admission, and it is not a pass.
    return label is Label.NOISE_CANDIDATE

freeze_request_allowed answers a narrow question: may someone file a request at all. Admission stays in the component you already allow to write a freeze. Every other label stops here.

The predicate below is illustrative. It is not a case from a live suite.

def order_total_monotone(order):
    priced = price(order)
    raw = sum(line.unit * line.qty for line in order.lines)
    assert priced.total >= raw
    assert priced.total == round(priced.total, 2)

Predicates that cannot fail will paint every cell green. assert priced.total == priced.total produces CLEAN and teaches nothing. This matrix will not catch that.

A mutation of the predicate will. If you do not already run one, do not read a green map as proof that the property has teeth.

Put the pins on the command line so shell history matches the record. Treat the flags as the contract, not as decoration.

python -m cell_triage \
  --base-rev "$BASE" \
  --patch-rev "$PATCH" \
  --fixture-locked fixtures/locked.json \
  --fixture-candidate fixtures/candidate.json \
  --seed 17 \
  --trials 25 \
  --repeats 3 \
  --properties props/order_total.py

Store one object per property. The following object shows the shape only. It is not output from an executed suite.

{
  "property": "order_total_monotone",
  "seed": 17,
  "trials": 25,
  "repeats": 3,
  "control": true,
  "patch_locked": false,
  "base_candidate": true,
  "patch_candidate": false,
  "label": "patch_defect",
  "freeze_request_allowed": false
}

How to read the map

Use one order so reviewers do not invent local dialects. The order is the review, not a suggestion beside it.

1. Confirm the seven pins are present, and that base_rev is the ancestor the patch claims to modify.
2. Read the control boolean before any other cell. If it is false, stop at CONTROL_BROKEN.
3. Apply the table. Do not collapse four booleans into a score.
4. On PATCH_DEFECT, FIXTURE_DRIFT, BOTH_AXES, or INTERACTION, return the diff. Freeze is out of scope.
5. On NOISE_CANDIDATE, attach the map to the inconclusive path you already run. Do not turn the attachment into a green check.
6. On CLEAN, merge or ask for a stronger property. Do not open a freeze in case the green was lucky.

A weighted score would imply a trade between a patch bug and a fixture bug. This gate does not offer that trade. Four bits are the whole result.

Where the patch is drafted and where the cells run

Drafting the candidate and replaying the four cells are different jobs. One needs a model. The other needs two revisions and two fixture files on disk.

Free model access can draft that patch, and a free server option can run the checkouts, when those options are currently offered. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The only product facts used here are those two operator-supplied availability claims. No model name, quota, hardware shape, duration, or continued availability is assumed.

A label is not stronger because the runner was unpaid. The model response is not an oracle. When generation returns an empty or partial diff, record a generation failure and leave the cells blank.

Do not invent passes to fill the grid. Read current terms before you depend on either option. Do not put the word free into a test assertion.

Availability is a workflow input, not a property of the patch. Budget the reruns with arithmetic, not with a borrowed benchmark. Four cells times three repeats is twelve suite executions per batch.

Run that batch when a freeze is requested, or when the diff touches fixture files. A documentation-only change does not need the grid. The extra runs would spend the pins and return a label you did not need.

Limitations, and who should skip the matrix

Two axes is the whole design. A bug shared by base and patch stays invisible unless the control cell goes red. A property you never wrote stays invisible even when every cell is green. The matrix isolates movement. It does not invent coverage.

Trials inside one cell can still flicker. The classifier consumes booleans, so write the reduction down. One defensible rule is that the cell is red if any trial failed. Store that sentence next to the seed.

Changing the reduction between runs voids the comparison, just as a seed change does. Keep the rule in the same record as generator_version. A later reader should not have to guess which reduction produced the booleans.

Skip the matrix when base cannot be checked out against both fixtures. A patch that deletes the locked fixture, or a suite bound to live services you cannot pin, will manufacture CONTROL_BROKEN or conceal drift. If generated patches currently merge with no locked digest, pin one fixture before you discuss freezes at all.

Skip it for documentation-only diffs, and for patches outside the behavior the property set claims to cover. Extra cells do not manufacture evidence. They multiply runtime.

If a freeze can still be recorded as a pass, fix that recording rule before you adopt this labeler. A precise map feeding a gate that treats inconclusive as green launders the same bugs. Cleaner paperwork does not change that outcome.

What the review note should contain

Lead with the label. Then the pins. Then the four booleans. A reviewer should recompute the label without opening a model transcript.

Explanation of how the patch was drafted comes after the map. It can be omitted when the label is already PATCH_DEFECT.

Require the map on the next agent patch that someone wants to freeze. Reject the request when the map is absent, when any pin is blank, or when the label is anything other than NOISE_CANDIDATE. If a model and a runner are how you will build that map, confirm the access terms still match what you are willing to accept, and keep the predicate in your own suite.