Quota Exhaustion Is a Failure Domain

Your free inference budget will run out mid-job. That is not bad luck. It is a failure domain.

Teams keep treating quota errors as billing noise. They wrap the call in a try block and move on. Then a four-step job dies at step three, and nobody knows what was already written.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free lane in this review is MonkeyCode's. The operator supplies free model access with 10 million tokens, plus a free server option. I have not benchmarked either one. I quote no latency, no success rate, and no throughput here, because I have no verified numbers for them.

The constraints you actually have

Three constraints shape any design around a free lane. Finite budget. Revocable access. Expensive redo.

Ten million tokens is a real amount of work. It is also gone in one afternoon if you pipe a monorepo through it. So stop treating the free tier as a floor you stand on. Treat it as a lane you drive in, with a sign that says the lane may close.

Access is revocable by design. I am not predicting anything about MonkeyCode's roadmap. I am stating a design assumption: no free lane belongs in your critical path.

Redo is the expensive part. Each replayed step costs tokens a second time. A resume that skips finished steps is cheaper than a retry that starts over.

The last week of DEV front-page posts was heavy on AI fatigue and tool churn. That matches production. The lane you use today is not the lane you use in six months. Design for the swap, not for the vendor.

Data flow

Keep the flow boring. Four boxes, one direction.

step list -> budget ledger -> lane adapter -> checkpoint log -> next step

The ledger charges before the call, never after. A timeout comes after the response, and it still spends tokens. If you charge on success only, your counter drifts low. Drift is worse than a hard stop, because you plan against a number that is wrong.

The checkpoint log is append-only. One JSON line per finished step. On restart, you read the log and skip what is done.

The adapter has exactly one method. That is the whole contract: complete(prompt) -> str. If leaving a lane takes three methods, you cannot leave it.

Where it breaks

Each row gets its own trigger and its own recovery. Do not collapse them into one except Exception. Name them, then test them.

The harness

Here is a runnable version. It uses the standard library only and never touches the network. Save it as lane_ledger.py.

# lane_ledger.py - Python 3.11+, standard library only.
import json, pathlib, tempfile

class QuotaExhausted(Exception):
    pass

class Ledger:
    """Charge before the call. Never after."""
    def __init__(self, budget: int):
        self.budget = budget
        self.spent = 0
    def charge(self, tokens: int) -> None:
        if self.spent + tokens > self.budget:
            raise QuotaExhausted(f"need {tokens}, left {self.budget - self.spent}")
        self.spent += tokens

class StubLane:
    """One adapter, one method. Swapping it means leaving the lane."""
    def __init__(self, ledger: Ledger, cost: int = 3):
        self.ledger = ledger
        self.cost = cost
    def complete(self, prompt: str) -> str:
        self.ledger.charge(self.cost)
        return "stub:" + prompt[::-1]

class JobRunner:
    """Append-only checkpoints. A resume skips finished steps."""
    def __init__(self, lane: StubLane, checkpoint: str):
        self.lane = lane
        self.path = pathlib.Path(checkpoint)
        self.executed: list[int] = []
    def done(self) -> set[int]:
        if not self.path.exists():
            return set()
        rows = [json.loads(l) for l in self.path.read_text().splitlines() if l.strip()]
        return {r["step"] for r in rows}
    def run(self, steps: list[str]) -> list[int]:
        self.executed = []
        done = self.done()
        for i, step in enumerate(steps):
            if i in done:
                continue
            out = self.lane.complete(step)
            with self.path.open("a") as f:
                f.write(json.dumps({"step": i, "input": step, "output": out}) + "\n")
            self.executed.append(i)
        return self.executed

def chaos_test() -> None:
    ckpt = str(pathlib.Path(tempfile.mkdtemp()) / "job.jsonl")
    steps = ["draft", "critique", "patch", "changelog"]
    ledger = Ledger(budget=7)
    runner = JobRunner(StubLane(ledger, cost=3), ckpt)

    try:
        runner.run(steps)          # 3 + 3 tokens, then the third call dies
    except QuotaExhausted:
        pass
    first = runner.executed
    assert first == [0, 1], first
    assert ledger.spent == 6, ledger.spent

    ledger.budget = 12             # top up, then resume
    second = runner.run(steps)
    assert second == [2, 3], second

    rows = [json.loads(l) for l in pathlib.Path(ckpt).read_text().splitlines()]
    assert [r["step"] for r in rows] == [0, 1, 2, 3], rows
    print("first run:", first, "tokens: 6")
    print("resume:", second, "tokens:", ledger.spent)
    print("checkpoints:", len(rows), "duplicate steps: 0")

if __name__ == "__main__":
    chaos_test()

Run it.

python lane_ledger.py
first run: [0, 1] tokens: 6
resume: [2, 3] tokens: 12
checkpoints: 4 duplicate steps: 0

Read that output like an architecture review. The first run spends six tokens and dies on the third call. The ledger refuses the call instead of overdrawing. The resume executes only steps two and three. Four checkpoints exist, and none repeats.

Now count where the lane's name appears in your own repo.

grep -rn "StubLane" --include="*.py" . | wc -l

One file is healthy. Ten files means your lane is load-bearing. That count belongs in the design review, not in a postmortem.

What the test does not prove

Be honest about scope. This harness proves resume logic, not inference quality. It never touches a network, a model, or a free server.

It cannot tell you how a real free lane behaves under load. It cannot promise that an allowance survives an account change. Treat the 10 million tokens as a starting allowance, not a guarantee.

The stub also always succeeds while it has budget. Real lanes can fail halfway through a stream. Your adapter still needs a partial-response path for that case, and this test does not cover it.

Who should not use this

Skip this approach if you need data residency guarantees. Prompts leave your host, and the checkpoint log writes model output to disk.

Skip it for latency-critical paths. A resume loop is slower than one direct call, by design.

Skip it if the work is regulated or audited. Pinning a model version for reproducibility is hard when you do not control that version.

What I would change next

Add a second adapter behind the same interface. Two lanes make the swap test real instead of theoretical. Then make chaos fail mid-stream, between tokens, not neatly between calls.

Move the ledger out of the process once more than one worker runs jobs. A file-based counter races under concurrency.

Log a stop reason for every halt. Quota exhaustion, rate limiting, and schema drift need different alerts and different humans.

The goal is not a reliable free lane. You cannot build one. The goal is a system that does not care which lane answers.

If you want to point this harness at a real endpoint, the free tier is a cheap place to start, and the operator supplies 10M tokens plus a free server option.