Why I Stopped Loading Entire Files Into Memory (And How It Saved My Web App)

When you build client-side tools, user privacy isn't just a feature. It's a strict architectural requirement.

No backend.
No uploads.
No server.

But nobody warns you about the hidden cost of running everything in the browser: Memory Limits.

I learned this the hard way while building ToolFlic, a suite of 100+ browser-based utilities.

When users started processing large PDFs and high-resolution images, the browser tabs started crashing.

Here is the story of why it happened, and how I fixed it.

The Naive Approach: Load Everything

When I first started building the file tools, my approach was simple.

I used the FileReader API to read the entire file, pass it to the processing library, and generate the output.

const reader = new FileReader();
reader.onload = (e) => {
  const buffer = e.target.result;
  // Process the entire buffer at once
};
reader.readAsArrayBuffer(file);

For a 2MB image, this works perfectly.

But what happens when a user uploads a 50MB PDF?

The browser tries to allocate all of that data into RAM at the same time.

The browser's garbage collector cannot keep up.

The UI freezes, and eventually, the tab crashes with the "Aw, Snap!" error.

The Reality of Client-Side Processing

JavaScript does not give you direct control over memory management.

You cannot manually free up an ArrayBuffer after you are done with it.

You have to rely on the browser's garbage collector, and it only runs when it wants to.

If you want to process large files without crashing, you have to stop thinking like a backend developer.

You have to think like a systems engineer.

The Solution: Chunking and Web Workers

I had to completely rewrite the way files were read and processed.

Here is the architecture that fixed it.

1. Read Files in Chunks

Instead of reading a 100MB file all at once, I read it in smaller chunks using File.slice().

const chunkSize = 5 * 1024 * 1024; // 5MB
let offset = 0;

while (offset < file.size) {
  const chunk = file.slice(offset, offset + chunkSize);
  const buffer = await chunk.arrayBuffer();

  // Process the chunk
  offset += chunkSize;
}

By reading in chunks, memory usage stays flat.

The browser only ever holds 5MB in memory at any given time, no matter how big the file is.

2. Offload to Web Workers

Even with chunking, heavy processing can block the main thread and freeze the UI.

I moved the processing logic into Web Workers.

Web Workers run on a separate thread.

The main thread stays free to handle user interactions, like dragging and dropping files.

3. The UI Feedback Loop

When you process large files client-side, the user needs to know the app hasn't frozen.

I used postMessage from the Web Worker to send progress updates back to the main thread.

// Inside the Web Worker
self.postMessage({ progress: 50, status: "Processing..." });

This allowed me to build a progress bar that updates in real-time.

The Tradeoff

Building things to run entirely in the browser is harder than sending a request to a server.

You have to respect the constraints of the user's device.

You have to handle edge cases like low memory and slow CPUs.

But the tradeoff is worth it.

By doing the work client-side, you guarantee that the user's data is completely private.

What I Learned

If you are building browser-based tools, treat the browser like a resource-constrained environment.

Never load the whole file at once.
Use Web Workers for anything CPU-intensive.
Always assume the user is on a low-end device.

You can see these techniques in action on ToolFlic - All Free Online Tools.
It handles large files entirely on your device, without ever hitting a server.

I'd love to hear how other developers handle memory management in the browser.

What techniques do you use?