OpenAI max_retries=2 Turned 6 Attempts Into 18 Requests
The openai Python SDK already retries a 429 twice by default, so the tenacity decorator from OpenAI's own rate-limit cookbook, stop_after_attempt(6), sent 18 requests per call to my test server, not 6. Pick one retry lay
The openai Python SDK already retries a 429 twice by default, so the tenacity decorator from OpenAI's own rate-limit cookbook, stop_after_attempt(6), sent 18 requests per call to my test server, not 6. Pick one retry layer: either OpenAI(max_retries=0) under a tenacity policy that only retries errors that waiting can fix, or drop tenacity and raise max_retries on the client.
This story was written with the assistance of an AI writing program. The numbers come from two runs on one Linux box: openai 3.51.0, tenacity 9.2.1 and Python 3.14.4, pointed through OPENAI_BASE_URL at a local stub that answers every chat completion request with an OpenAI-shaped error body and logs the SDK's x-stainless-retry-count header. None of this traffic went to OpenAI. Each run had twelve clients in parallel, one call each.
Two Retry Loops in One Call
The cookbook notebook How to handle rate limits builds the client as openai.OpenAI(api_key=...) and wraps the call in @retry(wait=wait_random_exponential(min=1, max=60), stop=stop_after_attempt(6)). Read alone, that is six attempts with random exponential backoff.
The client underneath has its own loop. The SDK README says: "Certain errors are automatically retried 2 times by default, with a short exponential backoff." The list includes 429 Rate Limit. In 3.51.0 that backoff starts at 0.5 s and doubles, minus up to 25% jitter. So each tenacity attempt is really three requests, and tenacity only sleeps between the groups.
18 Requests per Call, Three at a Time
These three lines from the stub's log are the cookbook code, run as published, against three errors: A1 got rate_limit_exceeded, Q1 got insufficient_quota, and U1 got a 401 for a wrong API key.
A1: 18 requests (HTTP 429) over 19.9s, x-stainless-retry-count 0,1,2,0,1,2,0,1,2,0,1,2,0,1,2,0,1,2
Q1: 18 requests (HTTP 429) over 20.6s, x-stainless-retry-count 0,1,2,0,1,2,0,1,2,0,1,2,0,1,2,0,1,2
U1: 6 requests (HTTP 401) over 20.3s, x-stainless-retry-count 0,0,0,0,0,0
Every cookbook call that got a 429, in both runs, looked like A1: 8 calls, 18 requests each, 12 of them SDK retries. Inside each group the third request went out 1.2 to 1.5 s after the first. The cookbook itself warns that "unsuccessful requests contribute to your per-minute limit, so continuously resending a request wonβt work". The inner loop resends twice within 1.5 s before tenacity's backoff even starts.
The Quota Error Got 18 Requests Too
OpenAI uses 429 for two different problems. rate_limit_exceeded clears up if you wait. insufficient_quota, "You exceeded your current quota, please check your plan and billing details", does not: no amount of backoff buys credits. Without an x-should-retry header, the SDK decides by status code, and tenacity's default is retry_if_exception_type(), which retries on any exception. So Q1 sent 18 requests and got the same answer each time.
U1 shows the other half. The SDK does not retry a 401, but tenacity does, so the wrapper still made 6 requests. Across both runs the four quota calls took 20.7 to 31.4 s to fail and the four bad-key calls 9.1 to 28.5 s. Both errors were final on the first response.
except openai.RateLimitError Never Ran
After the sixth attempt, tenacity does not raise the error it caught. With the default reraise=False it raises tenacity.RetryError, which wraps the last attempt. My client had except openai.RateLimitError and except openai.AuthenticationError handlers. All 12 cookbook calls across both runs skipped both and landed in the generic except Exception. Any code that maps a quota error to "check billing" or a rate limit to "try again later" never runs; RetryError.last_attempt.exception() holds the original error if you need it.
One Retry Layer That Knows Which 429 Is Which
This is the version I would ship. The client does not retry, tenacity retries only errors that waiting can fix, and reraise=True hands the original exception to the caller:
import openai
from tenacity import retry, retry_if_exception, stop_after_attempt, wait_random_exponential
client = openai.OpenAI(max_retries=0) # this layer does all the retrying
def is_transient(exc: BaseException) -> bool:
if isinstance(exc, openai.RateLimitError):
return exc.code != "insufficient_quota"
return isinstance(exc, (openai.APIConnectionError, openai.InternalServerError))
@retry(
retry=retry_if_exception(is_transient),
wait=wait_random_exponential(min=1, max=60),
stop=stop_after_attempt(6),
reraise=True,
)
def completion_with_backoff(**kwargs):
return client.chat.completions.create(**kwargs)
Against the same stub, the rate-limited call made 6 requests (8.4 to 21.5 s over two runs) and then openai.RateLimitError reached its own handler. The quota error and the bad key each made 1 request and failed in 0.1 s or less. In this SDK APITimeoutError is a subclass of APIConnectionError, so timeouts are covered too.
Or Let the SDK Be the Only Loop
If you do not need the quota distinction, the smaller fix is to delete tenacity and raise the SDK's own budget with OpenAI(max_retries=5). That made 6 requests per rate-limited call, retry counts 0 to 5, in 13.2 to 14.3 s, and the error came out as a plain openai.RateLimitError. According to the 3.51.0 source, it also honors a server's Retry-After of up to 120 s, which wait_random_exponential never reads; I did not measure that part. The cost: the SDK retries every 429, so the quota error also took 6 requests and 13.5 to 13.9 s to fail.
What This Test Does Not Show
The stub is not OpenAI. It answers instantly and sends no Retry-After or x-should-retry header, so the timings show backoff alone. The SDK obeys x-should-retry before it looks at the status code, and I have not checked whether OpenAI's real quota error sends one. I also cannot say how a real 429 burst moves your quota window, only how many requests your code sends into it.
Takeaways
- Count retry layers before you tune one. With the SDK default,
stop_after_attempt(n)means up to 3n requests. - Give retries one owner:
max_retries=0under tenacity, or the SDK alone with a largermax_retries. - Retry by cause, not by status code.
insufficient_quotaand a 401 should fail on the first response. - With tenacity, set
reraise=True, or yourexcept openai.RateLimitErrorhandlers never run. - Log
x-stainless-retry-counton your proxy, or setOPENAI_LOG=infoto see the SDK's own "Retrying request" lines.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.