Dev.to WebDev πŸ›  Dev πŸ‘ 0 πŸ“– 9 min read

Node.js Browser Delivery for Large CSV and PDF Files from Object Storage

Short answer: generate the CSV or PDF outside the request path, store the finished immutable object, and let the browser download it through a short-lived presigned URL; treat retries, byte ranges, URL expiry, and export

Short answer: generate the CSV or PDF outside the request path, store the finished immutable object, and let the browser download it through a short-lived presigned URL; treat retries, byte ranges, URL expiry, and export readiness as separate state transitions rather than stretching one Node.js request timeout.

That separation is the architecture decision. The application owns authorization and export state. Object storage owns the large byte stream. The browser owns resumption. Keeping those boundaries explicit prevents a slow download from consuming an application worker, but it also exposes a detail that teams often miss: a successful link response does not prove that the export is ready, that the URL will remain valid for the whole transfer, or that a retry will continue from the last byte.

This is an ADR, not a promise that presigned URLs cure every performance problem. They don't.

What should a Node.js browser export do when a large CSV or PDF download is slow?

Start by naming four invariants. An export identifier must refer to one logical result. A ready result must not change underneath a resumed download. Authorization to ask for the export is distinct from temporary authorization to fetch its bytes. Finally, the client must be able to distinguish "not ready," "link expired," and "byte transfer interrupted" because each condition demands a different action.

The failure boundary matters more than the timeout value. If Node.js generates a file and streams it during the same browser request, CPU work, database reads, serialization, proxy idle timers, and the user's network all share one critical path. Raising a 30-second timeout to 300 seconds merely gives every dependency a larger window in which to hold resources. It doesn't establish progress, durability, or resumability.

With an asynchronous export, the control path is small: create a job, poll its state, obtain a download capability after the object is complete, and refresh that capability when necessary. The data path is different: the browser transfers bytes from storage and may resume from a known offset. A presigned URL belongs to the data path, but issuing or refreshing it remains an authorization decision in the control path.

Don't conflate those paths.

This design also changes what "retry" means. Retrying export creation can duplicate expensive work unless the create operation has an idempotency key or the application can return the existing export identifier. Retrying the status request is cheap and should never restart generation. Refreshing an expired URL should preserve the export identifier and object identity. Retrying the byte stream should request only the missing range when the storage response proves that partial delivery is supported; if the object's identity has changed, the safe action is to discard the partial file and restart rather than splice bytes from two versions.

Record the invariants and failure boundaries

The useful state machine is deliberately boring:

queued -> generating -> ready -> expired

failed is terminal for that attempt, while a new attempt gets a new export identifier. The object should become visible to the download flow only after generation and upload are complete. A checksum, version identifier, or immutable object key can pin retries to the same bytes; which mechanism is available depends on the storage interface, so I'm not sure one universal validator can be prescribed without testing the actual service and browser path. The invariant is still clear: a resumed transfer cannot silently cross object versions.

Failures then land in one owned area instead of becoming a generic "download timed out" report:

Failure mode Owner Evidence to retain Correct next action
Generation exceeds its budget export worker export ID, phase, elapsed time fail the attempt; create a new attempt under policy
Browser asks before completion control API export state and poll count return state; do not issue a download URL
Presigned URL expires control API export ID and object identity reauthorize and issue a fresh URL
Transfer stops after some bytes browser and storage expected size, received offset, object validator resume the missing range when identity matches
Range cannot be satisfied browser requested offset and current object size discard stale partial data, refetch metadata, then restart
User loses access control API principal, policy decision, export ID refuse URL refresh even if an older link existed

Notice what isn't in the table: "retry everything." Broad retries can create duplicate exports, repeated database scans, and several concurrent downloads of the same object. Backoff reduces request frequency but cannot repair an ambiguous operation. Idempotency and state inspection have to come first.

For browser delivery, Content-Disposition controls whether content is presented inline or as an attachment and can carry a suggested filename. It is presentation metadata, not an authorization mechanism and not proof that the payload is a complete CSV or PDF. Use an ASCII fallback filename plus the extended filename form when non-ASCII names matter, and sanitize application-supplied names before constructing the header. The MDN reference documents the header syntax and browser behavior.

Compare the delivery shapes before tuning timeouts

There are three common shapes, and none wins every workload.

Shape Application resource cost Resume story Authorization boundary Suitable case Main limitation
Generate and stream in one request held for the full generation and transfer usually coupled to restarting the request one application request small, quick, rarely retried exports every dependency shares one timeout budget
Generate first, proxy bytes through Node.js application handles every downloaded byte application must implement and validate ranges centralized in the application policy requires mediation of every byte proxy bandwidth and connection lifetime remain on the critical path
Generate first, then issue a presigned object URL application leaves the bulk data path storage and browser can negotiate partial transfer short-lived capability after application authorization large immutable artifacts link lifetime, revocation policy, and cross-origin behavior require deliberate design

The third shape is the default decision for large CSV and PDF artifacts because it narrows the application critical path, not because it makes storage magically fast. The catch is capability leakage: anyone holding an unexpired presigned URL may be able to use it according to its scope. Keep the lifetime no longer than the operational need, avoid placing the URL in logs or analytics events, and require the control API to re-check current access before refreshing it.

It is not suitable when every byte must pass a synchronous policy check, transformation must occur during each read, or the client cannot reach the storage endpoint. In those cases, stick with an application or dedicated download proxy, accept its bandwidth and connection costs, and implement range semantics there. Direct streaming also remains valid for genuinely small exports whose generation has a tight, measured upper bound; adding a queue, worker, object lifecycle, and polling API to a 20 KB report would be operational theater.

Costs need the same decomposition. Stored bytes, write operations, read operations, and downloaded bytes may be priced separately, and retry behavior changes the latter two. Published pricing pages such as Backblaze B2's make those dimensions visible, but a pricing page is an input to a workload model, not evidence that one provider is universally cheaper. Model object retention, export frequency, abandoned exports, full restarts, range retries, and deletion policy with your own traffic distribution.

Put the critical path in code

The important code is not the storage SDK call. It is the decision function that refuses to issue a link before readiness, separates link refresh from job creation, and gives the browser stable object metadata. This Python sketch keeps infrastructure behind interfaces so the same state model can sit beside a Node.js control API without making the article a vendor setup guide:

from dataclasses import dataclass
from enum import Enum
from typing import Protocol


class ExportState(str, Enum):
    QUEUED = "queued"
    GENERATING = "generating"
    READY = "ready"
    FAILED = "failed"
    EXPIRED = "expired"


@dataclass(frozen=True)
class Export:
    export_id: str
    owner_id: str
    state: ExportState
    object_key: str | None
    size_bytes: int | None
    object_version: str | None


@dataclass(frozen=True)
class DownloadGrant:
    url: str
    size_bytes: int
    object_version: str


class ExportRepository(Protocol):
    def get(self, export_id: str) -> Export: ...


class ObjectSigner(Protocol):
    def sign_download(self, object_key: str, lifetime_seconds: int) -> str: ...


def authorize_download(
    *,
    export_id: str,
    principal_id: str,
    repository: ExportRepository,
    signer: ObjectSigner,
    lifetime_seconds: int,
) -> DownloadGrant:
    export = repository.get(export_id)

    if export.owner_id != principal_id:
        raise PermissionError("export is not available to this principal")
    if export.state is not ExportState.READY:
        raise RuntimeError(f"export is {export.state.value}")
    if export.object_key is None or export.size_bytes is None:
        raise RuntimeError("ready export has incomplete object metadata")
    if export.object_version is None:
        raise RuntimeError("ready export has no stable object identity")
    if lifetime_seconds <= 0:
        raise ValueError("lifetime_seconds must be positive")

    return DownloadGrant(
        url=signer.sign_download(export.object_key, lifetime_seconds),
        size_bytes=export.size_bytes,
        object_version=export.object_version,
    )

The public control API can map those domain outcomes to its documented response contract, but the browser logic should remain conservative. Create once. Poll until ready. Request a grant. On an interrupted transfer, retain the verified byte count and object identity; on retry, ask for the remaining bytes. If the link has expired, request a new grant for the same export and resume only after confirming the identity still matches. A 206 response indicates partial content, while a full 200 response to a resume attempt means the client must not append the entire representation to its partial file. An unsatisfied range can produce 416; record the offset and current size before deciding to restart.

Test the unhappy paths with a file large enough to outlive a convenient local request, then deliberately interrupt the connection at fixed offsets. Verify that only one export object is produced, a refresh does not regenerate it, a resumed file matches the original bytes, an authorization change blocks refresh, and expired artifacts are removed according to policy. Also test through the real CDN or proxy path β€” intermediate components can change headers and caching behavior β€” while keeping the test assertion about semantics rather than a particular vendor.

Observability should follow the two paths. For control requests, record export ID, state transition, generation duration, object size, authorization result, and grant issuance without logging the signed URL. For byte delivery, use storage or edge metrics for transferred bytes, partial responses, completion, and retry volume. Correlate with an opaque export ID, never the capability itself. A dashboard that shows only Node.js latency will look healthy after direct delivery and still miss users restarting multi-gigabyte downloads.

The acceptance criterion is end-to-end integrity.

Document the rejected option and its valid use case

The rejected default is "increase the Node.js timeout and keep streaming." It leaves generation and delivery in one failure domain, offers no durable ready state, and makes a network interruption indistinguishable from a generation problem at the user interface. That is a poor fit for large artifacts whose production and transfer times vary independently.

Still, rejection is contextual. Keep synchronous streaming when the output is small, cheap to reproduce, bounded by measurements, and subject to authorization that must remain active for the whole response. Keep a download proxy when network topology prevents direct storage access or when policy requires the application to mediate every read. Those choices cost more application capacity, but architecture is about preserving invariants under actual constraints, not collecting fashionable components.

For the asynchronous design, write down the operational limits before deployment: maximum export size, generation deadline, artifact retention period, presigned URL lifetime, maximum concurrent jobs per principal, retry ceiling, and the point at which a partial download is discarded. Your mileage may vary because browsers, proxies, and storage implementations differ. Measure with the same network routes and file sizes users encounter, and make the UI report the phase it actually knows: generating, ready, downloading, interrupted, or authorization required.

Then timeout tickets become diagnosable. More importantly, retries stop being guesses.

References

πŸ“° Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.