Your Safeguards Can Cancel Each Other Out: A Self-Test Matrix for Agent-Patch Gates

Property checks, fixture locks, and flaky freezes each work. Stacked in one gate, they can quietly undo each other.

That is the failure I keep finding in review. Every safeguard is individually defensible, and the composed gate still merges a patch that violates a property — because a freeze entry swallowed the test that would have proven it. The fix is not a fourth safeguard. It is a self-test: synthetic patches pushed through the full gate, with the expected verdict asserted for every combination.

The three laundering paths

A safeguard "launders" a failure when it converts a hard reject into an explainable one. Three shapes show up repeatedly.

None of these are bugs in the individual safeguard. They are ordering and keying bugs in the composition, which is exactly the layer nobody writes tests for.

The matrix: eight cases worth defending

Each row is a synthetic patch. The verdict column is what the gate must return — this is a specification, not a measurement of any real pipeline.

The two rows that matter most are property_plus_frozen and fixture_plus_frozen. A gate that consults the freeze ledger first returns hold for both. That is two escapes out of eight cases, produced by ordering alone.

Step 1 — Encode verdicts, not steps

Start from the decision table, not from the existing pipeline. The gate below is a reference implementation: no third-party packages, Python 3.10+, and every rule visible in one function.

# gate_selftest.py
from dataclasses import dataclass, field
from enum import Enum

class Verdict(str, Enum):
    PASS = "pass"
    HOLD = "hold:freeze"
    REJECT_FIXTURE = "reject:fixture"
    REJECT_PROPERTY = "reject:property"
    REJECT_TEST = "reject:test"

@dataclass
class PatchCase:
    name: str
    breaks_property: bool = False
    drifts_fixture: bool = False
    failing_test: str | None = None
    failing_assertion: str | None = None            # signature, e.g. "t_sum::eq"
    freeze: dict[str, str] = field(default_factory=dict)  # test -> signature at freeze time

def gate(case: PatchCase) -> Verdict:
    # Rule 1: fixture drift is never negotiable.
    if case.drifts_fixture:
        return Verdict.REJECT_FIXTURE
    # Rule 2: a failure is explained only by an exact signature match.
    explained = (
        case.failing_test is not None
        and case.freeze.get(case.failing_test) == case.failing_assertion
    )
    # Rule 3: property violations outrank any freeze.
    if case.breaks_property:
        return Verdict.REJECT_PROPERTY
    if case.failing_test and not explained:
        return Verdict.REJECT_TEST
    if explained:
        return Verdict.HOLD
    return Verdict.PASS

Rule ordering is the artifact. Rule 1 short-circuits before the ledger is read; Rule 3 short-circuits after explained is computed but before it is allowed to downgrade the verdict.

Step 2 — Require the matrix to fail your previous gate

A self-test that only passes is decoration. Keep a v0 gate that mirrors the pipeline you actually shipped, and assert the matrix rejects it.

def gate_v0(case: PatchCase) -> Verdict:
    # Freeze consulted first: the common shape when freezes predate property checks.
    if case.failing_test and case.failing_test in case.freeze:
        return Verdict.HOLD
    if case.drifts_fixture:
        return Verdict.REJECT_FIXTURE
    if case.breaks_property:
        return Verdict.REJECT_PROPERTY
    if case.failing_test:
        return Verdict.REJECT_TEST
    return Verdict.PASS

def run(matrix, expected):
    return {c.name: gate(c) for c in matrix}, {c.name: gate_v0(c) for c in matrix}

Wire it so CI fails in two directions: when gate disagrees with expected, and when gate_v0 agrees with expected on all eight cases. The second assertion catches the day someone "simplifies" the gate back into v0's ordering.

Step 3 — Generate the corpus instead of hand-writing it

Eight rows are a floor, not a target. Hand-written matrices rot; generated ones keep pace with the pipeline.

1. Enumerate the real knobs: property families, fixture hashes, freeze ledger entries, assertion signatures.
2. Ask a model for mutations of a known-good patch, one knob at a time, in the harness's PatchCase schema — not prose. Free model access is enough for this job; the output is a short structured record, not a long explanation.
3. Reject any generated case that does not compile into PatchCase. A mutant you cannot instantiate is not a test.
4. Deduplicate by (breaks_property, drifts_fixture, failing_assertion in freeze) so the matrix measures rules, not noise.
5. Run the harness somewhere cheap. It is pure Python with no services, so the free server option covers it without provisioning anything else.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Concretely, I use its free model access to emit PatchCase records and its free server option to run the harness; the gate logic, the ordering rules, and the expected verdicts are the work, and they run unchanged on any Python interpreter.

Step 4 — Adopt the three rules the matrix implies

If the matrix is too large to start today, take the rules it enforces and apply them by hand:

1. Freeze on signature, not name. Store the assertion signature at freeze time and compare it on every run. A changed signature is a new failure wearing an old exemption.
2. Fixture drift outranks everything. No ledger entry, waiver, or retry budget may excuse a fixture that moved with the patch.
3. Property results are terminal. A freeze may explain a flaky assertion; it may never explain a violated property. If the same test carries both, the property verdict wins.

These three rules are cheap to review and cheap to test. The expensive part is remembering that they interact.

Limitations and who should skip this

The matrix validates the gate's decision logic, not the gate's fidelity to production. If PatchCase drifts from what your pipeline actually emits, you get a green self-test over a fiction — re-derive the schema from the real job config whenever the pipeline changes.

It also assumes you have a freeze ledger with assertion-level detail. Teams that freeze whole files, or that have no ledger at all, will find flake_sig_drift unimplementable until they shrink the freeze scope. And if your pipeline has one safeguard rather than three, this is premature: get the single safeguard correct first, then test the composition.

Finally, do not use this as a substitute for a real regression suite. It answers one question — can my gate be talked out of a rejection — and nothing else.

Start with property_plus_frozen. If your gate returns hold, you already have the bug, and the rest of the matrix is just the receipt. If it returns reject:property, the useful thing to share back is the laundering case your gate still allows today.