Dev.to WebDev 🛠 Dev 👁 0 📖 3 min read

This rate limit says 10 per minute. It let 20 through in 6 seconds.

A rate limit of 10 requests per minute. Someone sends 10 requests at 0:57, then 10 more at 1:00. All 20 get through, in under 6 seconds. Nothing is broken. This is how the most common rate limiter works. Ho

This rate limit says 10 per minute. It let 20 through in 6 seconds.

A rate limit of 10 requests per minute. Someone sends 10 requests at 0:57, then 10 more at 1:00.

All 20 get through, in under 6 seconds.

A limit of 10 per minute. Burst B slides from 0:30 to 1:00. Once it crosses the minute edge, all 20 requests are allowed within 5.7 seconds

Nothing is broken. This is how the most common rate limiter works.

How a fixed window works

The simplest limiter cuts time into minutes and counts requests in each one. 12:00 to 12:01 is one window, 12:01 to 12:02 is the next. When a new minute starts, the count goes back to zero.

const counts = new Map(); // user -> { window, count }

function allow(user, now = Date.now()) {
  const window = Math.floor(now / 60_000); // which minute are we in?
  let c = counts.get(user);
  if (!c || c.window !== window) {
    c = { window, count: 0 }; // new minute, start from zero
    counts.set(user, c);
  }
  c.count++;
  return c.count <= 10;
}

(now is a parameter so you can test it with fake times.)

It's pretty much the first rate limiter everyone writes. One number per user, cheap to store, easy to explain.

Where it breaks

The counter doesn't remember what happened a second ago. It only knows which minute it is.

  • At 0:57 the first burst comes in. The count goes 1, 2, ... 10. All allowed.
  • At 1:00 the minute changes. The count resets to 0.
  • The second burst comes in. The count goes 1, 2, ... 10 again. All allowed.

So in about 6 seconds around the edge, the server got 20 requests. The limit says 10 per minute, and any 60 seconds that cover 0:57 to 1:03 have twice that.

That's the worst case: up to 2x the limit, if the timing lines up. And since every user's counter resets at the same moment, anyone who got blocked near the end of a minute retries at :00, all together.

The fix: count the last 60 seconds instead

Instead of asking "how many this minute?", ask "how many in the last 60 seconds?" every time a request comes in. This is called a sliding log. You keep the time of every allowed request and throw away the ones older than a minute.

const logs = new Map(); // user -> timestamps of allowed requests, oldest first

function allow(user, now = Date.now()) {
  const log = logs.get(user) ?? [];
  while (log.length && log[0] <= now - 60_000) log.shift(); // forget anything older than 60s
  if (log.length >= 10) return false;
  log.push(now);
  logs.set(user, log);
  return true;
}

Same traffic, switching between the two:

The same two bursts at 0:57 and 1:00. Fixed window allows all 20, sliding log allows 10 and rejects the second burst

With the sliding log, the second burst gets rejected. The first 10 requests are still inside the last 60 seconds, so there's no room. The next request would get in at 1:57, exactly 60 seconds after the first one.

I ran both functions on the same 20 timestamps. The fixed window allows 20. The sliding log allows 10.

So why does anyone use fixed windows?

Memory. The sliding log stores a timestamp for every allowed request. For 10 per minute that's nothing. For 1,000 per hour across a million users, it can be a billion timestamps.

A fixed window is one counter per user. That's also why it's so easy in Redis: INCR a key, let it expire after a minute, done.

Most real limiters sit somewhere in between. A sliding window counter keeps just two numbers, this minute's count and last minute's, and estimates from those. A token bucket allows short bursts on purpose but caps the average rate. Both handle the edge much better without storing every request.

The fixed window is fine when the window is long, like a daily quota. Nobody minds much if a customer squeezes two days of usage around midnight. It hurts when the window is short and its job is protecting a server.

You can drag the two bursts around yourself and watch when the limit holds and when it doesn't: https://software-atlas.haiduc0147.dev/t/rate-limiting#fixed-window

📰 Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.