Retrying flaky APIs in plain PHP: the 3-attempt loop behind ramrom.io's downloader

When I first shipped the downloader on ramrom.io, the resolve endpoint had no retry logic at all. One cURL call, one chance. If the provider answered "Service is busy", the visitor got an error page and I got nothing but a vague log line. Shared hosting, free tier provider API, flaky afternoons — it was failing several times a day.

The obvious fix was a retry loop, but retries done badly are worse than no retries. Fire ten requests at a provider that is already struggling and you are not being resilient, you are being a nuisance. So the loop has rules: at most three attempts, a short pause that grows between them, and a hard stop if the provider says something that a retry cannot fix.

The shape of it is simple. A function takes the URL to call and the headers to send, and runs a for loop from 0 to 2:

function provider_get($url, $headers, $max_attempts = 3) {
    $last_error = null;
    for ($attempt = 0; $attempt < $max_attempts; $attempt++) {
        $res = curl_request($url, $headers, 25); // 25s timeout
        if ($res['http_code'] === 200 && valid_payload($res['body'])) {
            return $res['body'];
        }
        $last_error = $res['http_code'] . ': ' . substr($res['body'], 0, 80);
        if ($attempt < $max_attempts - 1) {
            sleep($attempt + 1); // 1s, then 2s
        }
    }
    error_log('provider failed 3x: ' . $last_error);
    return null;
}

The pause matters more than the loop. sleep(1) then sleep(2) is nothing to the visitor — three seconds worst case — but it gives a congested provider breathing room instead of another instant hammer blow. On shared hosting, a few seconds of script runtime is cheap; spinning up Redis queues or a job worker is not an option, and for this workload it is not needed.

The important detail is deciding what not to retry. A 400 or a response that says the URL is unsupported is deterministic: retrying it three times wastes everyone's time and lies to the user with a longer spinner. So the loop only continues on the ambiguous failures — timeouts, connection resets, 5xx codes, and the provider's "Service is busy" body. Anything that reads like a real answer, even a "not found", exits immediately and the visitor gets an honest message.

I also made the failure message honest. Before, the user saw "Failed" and I could not tell you why. Now the endpoint returns a plain message like "The video provider is busy right now — please try again", and my server log records which attempt failed and why. That one-line log turned provider flakiness from a mystery into a metric I can read.

There was a second problem this loop quietly solved: the all-in-one provider I started with had busy spells that lasted minutes, not seconds. Three retries do not survive a five-minute outage, so on the third failure the site still tells the truth. But because each dedicated provider (Facebook reels, YouTube shorts, Instagram) now has its own retry loop, one provider's bad afternoon does not take down the other platforms. The failure is scoped to the platform that actually failed.

A few lessons from running this for a while:

• Cap the attempts low. Three is enough. Every retry beyond that was a longer wait for the same failure in my testing.
• Cap the timeout per attempt. Twenty-five seconds times three is already a minute. I would rather fail at ~40 seconds total than hang the page for five.
• Never retry a deterministic error. Check the body, not just the status code. A busy message retries; an unsupported-URL message does not.
• Log the last failure, not every failure. Three log lines per bad request is noise; one line with the final error is signal.

Nothing here is clever, and that is the point. On a plain PHP shared host with no queues, no Redis, and no build step, resilience is a for loop with a brain: a few attempts, growing pauses, honest exits, and logs that tell you what actually happened. The provider still says "Service is busy" some afternoons. Now it just does not ruin anyone's visit.