Retries Without Double Effects: Idempotency Keys for LLM Tool Calls

An LLM agent decides to call your create_refund tool. The request times out. The agent framework retries, or the model simply asks for the same tool call again on its next turn. Did the customer get one refund or two?

Timeouts don't tell you whether the first attempt ran. That was true for every HTTP client long before LLMs, and the fix is the same: make the side-effecting operation idempotent, so repeating it is safe. What changes with agents is how often the repeat happens and how hard it is to notice afterwards.

Why agents retry more than you expect

There are at least three layers that can repeat a tool call:

• your HTTP client or SDK, retrying on timeouts and 5xx responses;
• the agent loop, re-running a step after an error;
• the model itself, which may request the same action again if it didn't see a clear result.

Only the first one is usually configured on purpose. The other two show up in logs as "the agent did it twice".

The idea: one key per intended effect

The IETF draft The Idempotency-Key HTTP Header Field (draft-08, dated 22 October 2026; still an Internet-Draft, not an RFC) describes the common pattern. Its stated aim is to "make non-idempotent HTTP methods such as POST or PATCH fault-tolerant". The client sends a unique key with the request, and the server remembers what happened for that key.

The draft spells out the cases worth handling:

• the key "MUST be unique", and a UUID or similar random identifier is recommended;
• a key must not be reused with a different request payload, and the server should answer such a reuse with HTTP 422;
• if a request arrives while the first one with the same key is still processing, the server should answer with HTTP 409;
• the server should define a key expiration policy and document it.

Stripe's Idempotent requests documentation (no date shown; read 2026-11-05) shows one production policy. For its v1 API it says keys are removed after they are at least 24 hours old, and that retries with the same key return the saved response, "including 500 errors". It suggests V4 UUIDs and warns against putting sensitive data such as email addresses in keys.

Where the key comes from matters

For agents, the critical question is who generates the key. If the tool wrapper generates a fresh UUID on every call, a model-initiated repeat gets a new key and runs again. The key has to represent the intended effect, decided once.

  model --tool call--> agent loop --+--> tool wrapper --> API
                                    |        |
                        key decided |        | same key on
                        here, once  |        | every retry
                                    v        v
                              step record   server key store
                              (key, args)   (key -> result)

A practical rule: derive the key when the agent plans the action, store it with the step, and reuse it for every retry of that step. If the model asks for "the same refund" in a later turn, the loop can match it to the existing step instead of minting a new one.

A minimal server-side sketch

This in-memory version shows the three outcomes. A real one needs a durable store and an atomic "claim" operation.

import hashlib, json, threading

class IdempotencyStore:
    def __init__(self):
        self._lock = threading.Lock()
        self._rows = {}   # key -> {"fp": ..., "state": ..., "result": ...}

    @staticmethod
    def fingerprint(payload: dict) -> str:
        blob = json.dumps(payload, sort_keys=True).encode()
        return hashlib.sha256(blob).hexdigest()

    def handle(self, key: str, payload: dict, operation):
        fp = self.fingerprint(payload)
        with self._lock:
            row = self._rows.get(key)
            if row is None:
                self._rows[key] = {"fp": fp, "state": "running"}
            elif row["fp"] != fp:
                return 422, {"error": "key reused with different payload"}
            elif row["state"] == "running":
                return 409, {"error": "request with this key in progress"}
            else:
                return row["result"]          # replay, no second effect
        result = operation(payload)           # the side effect runs once
        with self._lock:
            self._rows[key].update(state="done", result=result)
        return result

And the tests that matter:

def test_retry_replays_instead_of_repeating():
    calls = []
    def refund(p):
        calls.append(p)
        return 201, {"refund_id": "r_1"}
    store = IdempotencyStore()
    first = store.handle("k1", {"order": 7, "amount": 500}, refund)
    again = store.handle("k1", {"order": 7, "amount": 500}, refund)
    assert first == again and len(calls) == 1

def test_same_key_different_payload_is_rejected():
    store = IdempotencyStore()
    store.handle("k1", {"amount": 500}, lambda p: (201, {}))
    status, _ = store.handle("k1", {"amount": 900}, lambda p: (201, {}))
    assert status == 422

Details that bite later

• Fingerprint the payload, not the raw bytes. Key order and whitespace differ between retries in some clients; a canonical form avoids false 422s.
• Decide what an error means. Replaying a saved failure (as Stripe's v1 API does) is predictable, but it means a transient failure won't be retried under the same key. Document which behavior you chose.
• Expire keys deliberately. An agent that resumes a paused task after the expiry window can repeat an effect. Pick the window with your longest realistic resume time in mind.
• Log the key with the step. When someone asks why two refunds appeared, the step record with its key is what tells you whether it was one intent retried or two separate decisions.

Where this fits

Being able to tell one decision retried from two decisions made is part of the problem we work on at Obligra. Obligra Verify™ is a system of record for AI-assisted decisions, in public preview, designed to retain the evidence your application selects. The idempotency pattern above is worth adopting regardless of where your records live.

This article was generated by an AI agent (Claude) for the Obligra Verify team and checked against the linked sources.