The Quota Wall: A Glossary and a Decision Tree for Agent Steps That Hit Free-Tier Limits

Consider this trace from a nightly agent job. It is a composite example, not a customer report, and the timings and counts are illustrative — but the shape is common.

03:40:02  step=summarize_prs    attempt=1  -> 200 OK   input_tokens=1180
03:40:07  step=write_changelog  attempt=1  -> 429      "rate limit: 20 rpm"
03:40:07  step=write_changelog  attempt=2  -> 429      (no sleep)
03:40:08  step=write_changelog  attempt=3  -> 429
03:40:08  job -> FAILED  "upstream error"

Three separate mistakes are packed into five seconds. Retries fired with no delay, the retry budget was spent on a condition that needed waiting rather than repeating, and the final error message threw away the only evidence that mattered — the response body.

Free model tiers make this failure mode routine, because "free" usually means "shared and rate-limited," not "unlimited." Below is the vocabulary I use to talk about it, a decision tree for choosing a response, and a dependency-free harness you can run in a few seconds against your own policy.

Glossary: six terms that decide everything

1. Admission — whether to send the request at all. Admission happens before the call: queue depth, tokens spent this hour, concurrency.
2. Soft throttle — a rejection that clears on its own within seconds or minutes. The classic 429 with a Retry-After header.
3. Quota wall — a limit that will not clear inside your deadline: a daily cap, exhausted credit, a disabled key. Retrying a quota wall is how a two-minute outage becomes an all-night loop.
4. Silent degradation — a 200 OK whose content is worse: truncated output, a dropped tool argument, a substituted smaller model. Status-code-only handling never sees it.
5. Replayability — whether the step can run again without duplicating a side effect (a comment posted twice, a ticket opened twice).
6. Deadline slack — remaining wall-clock budget minus the expected cost of one more attempt. When slack hits zero, every remaining branch is "fail."

Two derived terms get conflated and should not be: a tier switch moves the step to a different backend (another free model, a server you run yourself, a paid plan), while a retry means the same backend, later. Different costs, different failure modes.

The decision tree

Evaluate in order. The order is the policy; moving one branch changes behavior for every step.

[0] classify(response)  -------------------------- never decide on status code alone
 |
 +-- OK ...... commit
 |
 +-- SOFT_THROTTLE ... attempts left? slack > 5s? -- yes --> A: capped backoff + retry
 |                                                 \- no  --> D: fail loud (keep the body)
 |
 +-- TRUNCATED / SCHEMA_INVALID ... input over 60% of budget? -- yes --> B: degrade, retry once
 |                                                            \- no  --> D
 |
 +-- HARD_QUOTA ... has side effects? -- yes --> E: park for human
 |                  \- no ... replayable + tier left? -- yes --> C: switch tier, replay
 |                                                     \- no  --> D
 |
 \- UNKNOWN ... D: fail loud

Step 0 is classification, and it is the step people skip. Do it from three signals together: HTTP status, response body text, and finish_reason from the payload.

The artifact: quota_triage.py

Stdlib only, Python 3.10+, no network. Three parts: classify, decide, and a table-driven test with fake responses.

from __future__ import annotations
from dataclasses import dataclass, replace
from enum import Enum
import random

class Outcome(str, Enum):
    OK = "ok"
    SOFT_THROTTLE = "soft_throttle"
    HARD_QUOTA = "hard_quota"
    TRUNCATED = "truncated"
    SCHEMA_INVALID = "schema_invalid"
    UNKNOWN = "unknown"

def classify(status: int, body: str, finish_reason: str | None, parsed_ok: bool) -> Outcome:
    b = (body or "").lower()
    if status == 402:
        return Outcome.HARD_QUOTA
    if status == 429:
        # heuristic: some providers also use 429 for the daily cap
        return Outcome.HARD_QUOTA if ("daily" in b or "quota" in b) else Outcome.SOFT_THROTTLE
    if status in (401, 403):
        return Outcome.HARD_QUOTA if "quota" in b else Outcome.UNKNOWN
    if finish_reason == "length":
        return Outcome.TRUNCATED
    if 200 <= status < 300:
        return Outcome.OK if parsed_ok else Outcome.SCHEMA_INVALID
    return Outcome.UNKNOWN

The context object and the policy. Note that park_for_human is checked before the tier switch: a write with side effects never rides a replay branch, even when a second backend is idle.

@dataclass(frozen=True)
class Step:
    attempts_left: int
    input_tokens: int
    token_budget: int
    tiers_left: int            # e.g. 1 = one free-server leaf not tried yet
    replayable: bool
    has_side_effects: bool
    slack_s: float

def decide(o: Outcome, s: Step) -> str:
    if o is Outcome.OK:
        return "commit"
    if o is Outcome.SOFT_THROTTLE and s.attempts_left > 0 and s.slack_s > 5:
        return "backoff_retry"
    if o in (Outcome.TRUNCATED, Outcome.SCHEMA_INVALID) \
       and s.input_tokens > 0.6 * s.token_budget and s.attempts_left > 0:
        return "degrade_then_retry"
    if o is Outcome.HARD_QUOTA and s.has_side_effects:
        return "park_for_human"
    if o is Outcome.HARD_QUOTA and s.replayable and s.tiers_left > 0:
        return "switch_tier_replay"
    return "fail_loud"

def backoff_delay(attempt: int, base: float = 1.0, cap: float = 30.0) -> float:
    raw = min(cap, base * (2 ** attempt))
    return raw * (0.5 + random.random() / 2)   # jitter to 50-100% of raw

The test table is the part worth copying. It costs nothing to run and it fails loudly when your policy drifts.

BASE = Step(attempts_left=2, input_tokens=1200, token_budget=8000,
            tiers_left=1, replayable=True, has_side_effects=False, slack_s=90)

CASES = [
    # status, body,                  finish, parsed_ok, step,                                      want
    (200, "",                       "stop", True,  BASE,                                      "commit"),
    (429, "rate limit: 20 rpm",     None,   False, BASE,                                      "backoff_retry"),
    (429, "daily quota exceeded",   None,   False, BASE,                                      "switch_tier_replay"),
    (429, "daily quota exceeded",   None,   False, replace(BASE, has_side_effects=True),      "park_for_human"),
    (200, "",                       "length", True, replace(BASE, input_tokens=5200),         "degrade_then_retry"),
    (200, "",                       "stop", False, replace(BASE, input_tokens=5200),         "degrade_then_retry"),
    (429, "rate limit: 20 rpm",     None,   False, replace(BASE, slack_s=2),                  "fail_loud"),
    (500, "internal",               None,   False, BASE,                                      "fail_loud"),
]

if __name__ == "__main__":
    fails = 0
    for status, body, fin, ok, step, want in CASES:
        got = decide(classify(status, body, fin, ok), step)
        if got != want:
            fails += 1
            print(f"FAIL {status} {body!r:28} -> {got} (want {want})")
    print(f"{len(CASES) - fails}/{len(CASES)} policy cases pass")
    raise SystemExit(1 if fails else 0)

Run it with python quota_triage.py. Two caveats: the harness proves your policy, not your provider — it never touches the network — and the Step numbers are placeholders you should replace with your job's real budget.

Decision table: what each leaf costs when it is wrong

Worked leaves

Leaf A — soft throttle: wait once, with a stated cap

backoff_delay(0) returns 0.5–1.0s; backoff_delay(4) returns 15–30s. Cap at three attempts and log the delay you actually slept, so a postmortem can tell "the provider was slow" apart from "our backoff was too short."

Leaf B — degraded payload: shrink before you repeat

Illustrative arithmetic: a step with a 10,000-token budget sent 8,600 tokens and came back with finish_reason=length. Since 8600 > 0.6 * 10000, degrade_then_retry fires. Degrade means drop optional tool definitions, cut retrieved context to top-k, and lower the output max_tokens. One retry, then fail_loud.

Leaf C — quota wall: switch tiers instead of sleeping

switch_tier_replay is the leaf where a second free backend earns its place: the same step runs against a different quota pool, so the replay is testable without a credit card. This is where MonkeyCode's free model access and free server option fit my workflow — two separate pools to alternate between, with the harness above deciding when to move. Treat availability as operator-supplied and re-read the project page for today's limits; any number quoted in a blog post ages badly.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Leaf D — side effects: park, do not replay

If a step writes — posts, files, charges — a quota wall returns park_for_human even when replayable=True, unless the write carries a dedupe key. The queue entry should hold the request body hash, the attempt count, and the raw error string.

Limitations and who should not use this

• Body-text sniffing is a heuristic. Providers reword errors; keep classify as a patchable seam and log raw bodies for a week after any provider change.
• 429 is not standardized. Some backends use it for daily caps, some for per-minute limits, some for concurrency.
• Latency, output-quality drift between tiers, and cost are out of scope for the harness.
• "It worked on the second backend" is not a dedupe strategy. Never replay a non-idempotent write on a tier switch.

Skip this approach if your p99 requirement is tighter than your retry budget, if you need a contractual uptime guarantee, if your data cannot leave a region you can verify, or if writes dominate the workload. A decision tree does not fix a shared queue.

A five-minute check before trusting any policy

1. Paste three real error bodies into CASES. Real strings beat ten guesses.
2. Run the harness. Every failing case is a policy you were already running silently.
3. Set two hard caps: attempts per step, and steps per job.
4. Log outcome, attempt, delay_slept, and the raw body on every failure.

If you want to exercise Leaf C before spending anything, wire the harness to a second free pool first — the MonkeyCode project page is where I would start for that, and the disclosure above is the only reason it appears here.