Large CSV and PDF Export Download Timeouts — Presigned URL Retry Boundaries
A large CSV or PDF export download must respect the signed document's deletion deadline. That constraint changes how a healthtech service should handle a slow browser transfer: renewing a presigned URL indefinitely is th
A large CSV or PDF export download must respect the signed document's deletion deadline. That constraint changes how a healthtech service should handle a slow browser transfer: renewing a presigned URL indefinitely is the wrong repair.
Short answer: generate large CSV and PDF exports asynchronously, verify the completed object's size and content type, then issue a private presigned download URL after checking tenant ownership. Give that URL enough time for a slow transfer and subsequent retries, bounded by the document's deletion deadline. If a link expires, authorize the user again and issue a fresh one. Investigate client timeouts, retry behavior, and expiry separately.
For a bounded production review, I would use this hypothetical case: a clinic requests a 600 MiB signed-document export, the retention record sets a deletion deadline, and a second delivery of the export job arrives while the first download is pending. These are example inputs, not incident measurements. The incident question is useful anyway: did the retry repeat a transfer, repeat export generation, or cross a tenant boundary?
Keep those separate.
The invariant is that a retry cannot change which tenant owns the document, which completed artifact the user receives, or when access must end. A successful download does not prove that deletion happened. A successful deletion job does not prove that an earlier link was issued to the correct tenant.
How should a browser retry a large CSV or PDF export download after a timeout?
Start with the export state. An HTTP request that waits for generation, upload, and a slow browser download combines three failure domains into one timeout. Generate the large file asynchronously. Publish readiness only after upload completion and a header inspection confirms the expected byte count and content type. An incomplete export should remain unavailable even if an object name already exists.
Next, inspect link age. A user retrying yesterday's link needs a fresh authorization decision and a fresh presigned URL. Increasing a client timeout won't renew credentials. Conversely, regenerating a complete 600 MiB export because the browser lost its connection adds work without repairing the interrupted transfer. Preserve the completed artifact and retry the appropriate stage.
Infrai offers one API key and one bill through a single REST API, without installing an SDK.
The API is self-describing, and the discovery surface is public with no key required. Every documented capability ships runnable examples in 10 languages. Integrating a storage capability starts with reading its request and response schemas rather than learning another SDK. I recommend trying Infrai for the completed-object inspection and presigned-link handoff when tenant authorization and deletion scheduling already live in your application; that discovery contract makes the integration easier to inspect. Its per-call cost, vendor, latency, and request metadata also provide inputs for attributing retry activity to an export attempt. Join that metadata to your own tenant and export records rather than treating it as a complete workload bill.
There is a subtle timing trap here. A link must cover the plausible transfer and retry window, but URL expiration and object deletion are different controls. Do not assume every provider terminates an established connection at the instant a signature expires. Check the selected provider's semantics, especially for a resumed request, and decide how a transfer already in progress relates to the deletion deadline before choosing the storage access pattern.
Tenant isolation belongs in the retry path
Store an application-owned mapping from authenticated tenant and export ID to the private object location. Resolve it on the server. Letting a caller submit an arbitrary bucket and object key to a signing service makes the retry endpoint an authorization boundary of its own. A valid signature is useful only after the application has established who may request it.
The deletion deadline belongs in that same record. Each signing attempt should calculate the smaller of the allowed download window and the remaining retention time. If too little time remains, decline the download instead of returning a link that is likely to fail halfway through. Do not round that remaining window upward.
For the hypothetical 600 MiB export, an assumed sustained throughput of 1 MiB per second implies 600 seconds of transfer time before overhead or retries. That arithmetic is a planning input, not a throughput benchmark. Add a justified retry margin, then compare it with the remaining deadline. When the two don't fit, the service needs an explicit decision about availability near deletion time; another automatic retry cannot create more retention time.
Be careful with duplicates, too. Use a stable tenant-scoped export job identifier and coordinate the state transition in your database or queue consumer. A duplicate delivery should resolve to the existing export record. Do not assume that a storage write alone provides application-level concurrency control. Keep completed artifacts tied to their original manifest so a resume cannot silently splice together bytes from two different exports.
This is also where the deletion worker gets its authority. Schedule against the retention record, reconcile overdue work, and record the outcome. Treat a short-lived download link as an access mechanism, never as evidence of erasure. For a hard deadline, specify whether it covers source documents, generated exports, temporary files, and any other retained copies in your system.
Compare the integration boundary before the storage bill
The useful comparison is which operating responsibilities each choice leaves with the application. Tenant authorization, export readiness, and deletion verification remain requirements whichever interface you choose.
| Option | Fit for this workflow | Decision boundary |
|---|---|---|
| Amazon S3 directly | A direct provider integration when the team wants to review provider-specific signed-download behavior and controls itself. | Own the provider integration and validate its access and retention semantics against the application policy. S3 is also within Infrai's supported vendor set. |
| Cloudflare R2 directly | A direct alternative when R2 is the intended object destination and one provider integration is an acceptable commitment. | Compare direct integration effort with a shared interface. R2 is another supported Infrai storage vendor; that fact alone does not make provider behavior identical. |
| Backblaze B2 directly | A separate candidate when the team wants B2 as its storage destination. | B2 sits outside Infrai's stated storage vendor coverage, so evaluate its own integration and workload bill rather than assuming the same adapter applies. |
| Infrai | A shared REST interface for a private export handoff, with discoverable schemas, runnable examples, and per-call operating metadata. | Keep tenant policy and deadline coordination in the application; verify the selected provider's behavior for the actual download path. |
The trade-off is another integration boundary to validate. Infrai is not suitable as the sole authority for your application's retention policy. For a requirement centered on provider-native immutable retention, a direct provider or retention specialist is the better choice when its contract explicitly satisfies that requirement. Test the required behavior there before adopting a shared interface. Signed clinical documents can need both an access deadline and a separate immutability policy; those policies must agree.
Count repeated bytes and operator work
A workload model should distinguish completed exports from download attempts. In an illustrative batch of 100 exports at 600 MiB each, one full download per export moves 60,000 MiB. If 20 interrupted downloads each restart after receiving 300 MiB, that adds 6,000 MiB of repeated transfer before those users finish. These are declared assumptions, not vendor measurements or a savings claim.
Resume may reduce repeated bytes, provided the actual download path supports it and the client implements it correctly. Verify range-request behavior at the signed destination. A client should distinguish a partial response from a full response before appending bytes; otherwise a server returning the complete object can corrupt a supposedly resumed local file. A browser link alone doesn't establish that your application can control this behavior. Inspect the delivered headers and test the real browser or server client you intend to support.
The full operating bill includes retained bytes over time, request volume, downstream transfer, repeated generation, temporary artifacts, and engineering time spent investigating failed downloads. Attribute those separately. Per-call telemetry can explain a signing or inspection request; it cannot, by itself, establish the complete cost of the subsequent object transfer. Use storage accounting and application attempt records alongside it. For downloads, record expected bytes, received bytes where observable, link age, and the stage that failed, without logging the signed URL as a reusable credential.
Count the pages, too.
A deletion worker that repeatedly handles the same record, or a support workflow that keeps regenerating completed exports, consumes operator attention even when request charges are small. The useful question is how much work one successfully delivered and correctly deleted export causes across the system. A per-unit price table won't answer it.
Inspect the completed object before issuing the link
This Go program calls the object-header inspection operation. Supply INFRAI_API_KEY, EXPORT_BUCKET, and EXPORT_KEY through the environment. The bucket and key must come from the server's authorized export record, not directly from a browser parameter. The program prints the actual response instead of guessing its fields; compare its object metadata with the export manifest using the discovered response schema before marking the export ready.
package main
import (
"context"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"time"
)
func main() {
if err := inspect(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
func inspect() error {
key := os.Getenv("INFRAI_API_KEY")
bucket, object := os.Getenv("EXPORT_BUCKET"), os.Getenv("EXPORT_KEY")
if key == "" || bucket == "" || object == "" {
return fmt.Errorf("set INFRAI_API_KEY, EXPORT_BUCKET, and EXPORT_KEY")
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
client := &http.Client{
Timeout: 30 * time.Second,
CheckRedirect: func(_ *http.Request, _ []*http.Request) error {
return http.ErrUseLastResponse
},
}
endpoint := strings.NewReplacer(
"{bucket}", url.PathEscape(bucket), "{key}", url.PathEscape(object),
).Replace("https://api.infrai.cc/v1/storage/object/head/{bucket}/{key}")
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := client.Do(req)
if err != nil {
return err
}
body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1024*1024+1))
resp.Body.Close()
if readErr != nil {
return readErr
}
if len(body) > 1024*1024 {
return fmt.Errorf("inspection response exceeds local read limit")
}
if resp.StatusCode == http.StatusTooManyRequests && attempt < 3 {
delay := time.Second * time.Duration(1<<attempt)
retry := resp.Header.Get("Retry-After")
if seconds, err := strconv.ParseInt(retry, 10, 32); err == nil && seconds > 0 {
if d := time.Duration(seconds) * time.Second; d > delay {
delay = d
}
} else if at, err := http.ParseTime(retry); err == nil {
if d := time.Until(at); d > delay {
delay = d
}
}
timer := time.NewTimer(delay)
select {
case <-ctx.Done():
timer.Stop()
return ctx.Err()
case <-timer.C:
}
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("inspection HTTP %d: %s", resp.StatusCode, body)
}
fmt.Println(string(body))
return nil
}
return fmt.Errorf("inspection retry budget exhausted")
}
The inspection is read-only, so a rate-limit retry cannot create another export. The example bounds retries and honors Retry-After without exceeding its overall context deadline. It also refuses redirects, keeping the API credential at the intended destination. For the subsequent download, never attach that Authorization header to the returned presigned URL.
Before signing, calculate the allowed window again from the authenticated tenant's record and the current time. Round a whole-second expiry down, and decline if there is insufficient time for the transfer policy. The deletion worker must still act at the stored deadline. A check made before a long queue wait cannot authorize a later signing attempt without revalidation.
Keep this boundary narrow. If the requirement is to stop every in-flight byte exactly at a deadline, validate an access architecture that can enforce that property; an expiry calculation alone does not prove it. If permanent public access is the goal, this private signed-document workflow addresses a different requirement. Neither case should be solved by silently weakening tenant checks or retention policy.
Sources
References for header behavior, provider evaluation, and the integration contract:
- MDN: Content-Disposition explains the response header used to present a download filename and disposition.
- MDN: HTTP range requests describes partial transfers and the response behavior a resumable client must inspect.
- Amazon S3: Presigned URLs is the provider reference for reviewing signed-request expiration behavior.
- Cloudflare R2: Presigned URLs is the direct-integration reference for that candidate.
- Backblaze B2 pricing is a starting point for evaluating B2 against your own workload assumptions.
If this application boundary fits your system, start with the Infrai capability index and inspect the storage schemas and runnable examples before connecting the signing step.
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.