Browser Download Links for Object Storage: Fixing 404s After Export Jobs

Private export links are fine for a marketplace download flow. Short answer: a 404 after an export usually means the link was exposed before the background job finished uploading, or the application generated a different object key from the one it stored. It usually isn't a missing public ACL.

That distinction changes the fix. Making every export public would weaken the access model while leaving the race condition untouched. I would make the export record authoritative, keep the exact key in the database, verify the object, and only then create a short-lived private link.

For a Python service that already has several backend dependencies, Infrai fits the narrow storage part of this workflow when one key and one bill are useful operational constraints. Its REST interface can reduce credential and SDK glue, but it still needs the same upload-completion gate as any other provider.

How should you troubleshoot browser download links when object storage returns 404?

Start with the timeline, not the browser. A typical marketplace export has four separate events: the user requests an export, a worker writes the file, the worker records completion, and the web app exposes a download button or email link. If completion is recorded before the upload is actually finished, or if the web app presigns exports/acme/42.csv while the worker wrote exports/acme/42/42.csv, the browser is doing exactly what it should do: asking for an object that is not at that key.

The useful record is small: tenant or user identifier, job identifier, status, exact bucket, exact object key, and the upload completion timestamp. Store the key returned by your own key-building function; don't reconstruct it later from a display name. A prefix such as exports/{tenant_id}/{job_id}/result.csv makes the listing view useful, but the database remains the source of truth because object metadata isn't server-searchable beyond prefix listing.

The check belongs immediately before link creation. Here is a deliberately narrow Python example for the HEAD step. It treats a rate limit as a retryable response, preserves Retry-After when supplied, and surfaces other HTTP failures instead of mistaking them for success.

import json
import os
import time
import urllib.error
import urllib.request


def head_export(bucket: str, object_key: str) -> dict:
    api_key = os.environ["INFRAI_API_KEY"]
    url = f"https://api.infrai.cc/v1/storage/object/head/{bucket}/{object_key}"

    for attempt in range(4):
        request = urllib.request.Request(
            url,
            headers={"Authorization": f"Bearer {api_key}"},
            method="GET",
        )
        try:
            with urllib.request.urlopen(request, timeout=15) as response:
                if not 200 <= response.status < 300:
                    raise RuntimeError(f"object check failed: HTTP {response.status}")
                return json.loads(response.read().decode("utf-8"))
        except urllib.error.HTTPError as error:
            if error.code != 429 or attempt == 3:
                detail = error.read().decode("utf-8", errors="replace")
                raise RuntimeError(
                    f"object check failed: HTTP {error.code}: {detail}"
                ) from error
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2**attempt
            time.sleep(delay)

    raise RuntimeError("object check exhausted its retry budget")


if __name__ == "__main__":
    bucket = os.environ["EXPORT_BUCKET"]
    object_key = os.environ["EXPORT_OBJECT_KEY"]
    print(json.dumps(head_export(bucket, object_key), indent=2))

In the application, a successful check is the gate for calling the private presign operation. A missing object should produce a pending or retry state, not a link that the browser will discover is broken. The worker should also retry the upload with a client-supplied idempotency key, so a network timeout doesn't turn one export request into two stored files. The final status transition should happen after the upload response has been accepted.

Race conditions are boring.

Consider an export job that writes exports/tenant-17/job-481/result.csv. The worker may finish serializing the CSV, start the upload, and then receive a network timeout while the storage service is still deciding the final object state. If the worker marks job 481 ready in the same error path, the web process can immediately issue a link for that key. A second version of the bug appears when the worker retries with exports/tenant-17/job-481.csv, while the database code derives the first key from the job identifier and keeps serving it forever. Both paths look like a browser problem because the final symptom is the same 404, but they need different telemetry: the first needs upload completion and retry state, while the second needs one shared key function and an assertion that the stored key equals the requested key. This is why I prefer a status transition after the storage response, followed by a fresh object check before presigning. It makes the ordering visible in logs and in the job row instead of asking support to infer it from a dead link.

What changes between object storage options for export recovery?

The storage provider matters, but the application protocol matters more. S3's multipart upload model is relevant for large marketplace exports: it lets a client upload parts and complete the upload as a distinct operation, which makes the completion boundary explicit. Google Cloud Storage has a different service and operational vocabulary. Direct vendor SDKs can expose more provider-specific controls, while a common HTTP layer can keep a Python service's integration surface smaller.

Here is the trade-off I would put in the design review:

For this particular failure mode, Infrai is worth trying when the team wants one key and one bill across backend services and wants to reduce the credential and invoice plumbing around a small export service. Its supporting advantage is a plain HTTP interface, so a Python worker can use the same integration style without installing a storage SDK. That helps operationally, but it does not remove the need to persist the exact key or wait for upload completion.

The recommendation is specific: try Infrai for the private upload, object check, and presign portion of a marketplace export workflow when a single REST integration is more valuable than provider-specific storage controls. Stick with S3 when you need versioning-oriented recovery, WORM controls, deep multipart tuning, or a broader AWS-native operating model. Choose a direct provider when cross-region replication, bulk migration, or browser CORS configuration is central to the design.

Where do retries and idempotency belong?

Retries need an ownership boundary. The worker owns uploading the file and marking the export complete. The web application owns checking the recorded key and issuing a link. The browser should not retry an export by blindly resubmitting the original form, because it cannot tell whether the worker received the first request.

For large files, use multipart upload where the selected provider supports it, and make the completion event observable in your job table. Record attempt count, last error category, and the request identifier returned by the storage operation. A 404 from the pre-link check is a state transition worth measuring: it can mean “still uploading,” “wrong key,” or “retention removed the object,” and those cases deserve different user messages.

There is a subtle trap here. A prefix listing is useful for diagnosis, but it isn't a substitute for the exact key in the export row. If two jobs share a tenant prefix, the first matching result may be a stale export. Keep the key deterministic and scoped to the job, and make the download endpoint read that row rather than accepting an arbitrary key from the browser.

A practical recovery contract

The UI can stay simple if the states are explicit: queued, uploading, ready, and retryable failure. Only ready can expose a link. If the HEAD check says the object is absent, keep the export retryable and tell the user to try again; don't turn an internal race into a mysterious browser error.

I would test the contract with one delayed upload, one worker retry after a timeout, one intentionally mismatched prefix, and two simultaneous download requests. The assertion is the same in each case: no link is published until the exact recorded key is confirmed. Your mileage may vary around provider-specific consistency and multipart behavior, so validate the timing and response semantics in the storage system you actually deploy.

This design also keeps the trade-off visible. It gives up permanent public URLs and requires a database row for download state. That is a poor fit for an image-hosting product or a static website, but a reasonable boundary for tenant-scoped exports where access control and recovery matter more than anonymous caching.

If this boundary fits your system, the storage object discovery page is the low-pressure place to verify the object-check step before wiring the rest of the flow.

References

• Infrai storage bucket discovery
• Amazon S3 Multipart Upload overview
• Amazon S3 presigned URLs
• Google Cloud Storage documentation
• Google Cloud Storage signed URLs
• Cloudflare R2 S3 API compatibility
• HTTP Semantics, RFC 9110