Building a Rate Limiter in Go: Token Bucket vs Sliding Window

Rate limiting is one of those features that looks simple until you're paged at 2 am because a client is hammering your API and taking down the service. Adding it after the fact is always more painful than doing it right the first time — so this post covers two practical algorithms, when to pick each, and how to implement both cleanly in Go.

Why Rate Limiting Matters

Without rate limiting, a single misbehaving client can exhaust your CPU, database connections, or third-party API quota. This is both a reliability problem and a security vector: credential stuffing, brute-force attacks, and scraping bots all rely on the absence of effective limits. Any production API exposed to the internet should have rate limiting in place before the first deploy.

A secondary concern is fairness: in a multi-tenant service, one noisy client should not degrade the experience for everyone else. Rate limiting enforces that contract.

Algorithm 1: Token Bucket

The token bucket algorithm is the most common choice. The idea is straightforward: a bucket holds up to capacity tokens. Tokens are refilled at a fixed rate. Each request consumes one token. If the bucket is empty, the request is rejected.

Why it is popular: it naturally handles burst traffic. A client that has been idle can consume several tokens at once up to the bucket capacity, then gets throttled to the refill rate. This matches how most real-world clients behave — SDKs retry with backoff, mobile apps batch operations, and occasional bursts are expected.

Here is a thread-safe token bucket in pure Go with no external dependencies:

package ratelimit

import (
    "sync"
    "time"
)

type TokenBucket struct {
    mu       sync.Mutex
    tokens   float64
    capacity float64
    rate     float64 // tokens per second
    lastTime time.Time
}

func NewTokenBucket(capacity, ratePerSec float64) *TokenBucket {
    return &TokenBucket{
        tokens:   capacity,
        capacity: capacity,
        rate:     ratePerSec,
        lastTime: time.Now(),
    }
}

func (tb *TokenBucket) Allow() bool {
    tb.mu.Lock()
    defer tb.mu.Unlock()

    now := time.Now()
    elapsed := now.Sub(tb.lastTime).Seconds()
    tb.lastTime = now

    tb.tokens = clamp(tb.capacity, tb.tokens+elapsed*tb.rate)

    if tb.tokens < 1 {
        return false
    }
    tb.tokens--
    return true
}

func clamp(max, val float64) float64 {
    if val > max {
        return max
    }
    return val
}

The catch: the token bucket is permissive by design. A client with a full bucket can fire a significant spike in a very short window. For APIs where consistent throughput ceilings matter more than burst tolerance — auth flows, payment endpoints, account recovery — that can be a problem.

Algorithm 2: Sliding Window Counter

The sliding window algorithm provides smoother, stricter enforcement. Instead of a bucket, it tracks request counts in a rolling time window. The common approximation uses two fixed buckets (current window and previous window) and interpolates based on how far into the current window you are — accurate enough for rate limiting without storing every individual timestamp.

package ratelimit

import (
    "sync"
    "time"
)

type SlidingWindow struct {
    mu          sync.Mutex
    limit       int
    windowSize  time.Duration
    currCount   int
    prevCount   int
    windowStart time.Time
}

func NewSlidingWindow(limit int, window time.Duration) *SlidingWindow {
    return &SlidingWindow{
        limit:       limit,
        windowSize:  window,
        windowStart: time.Now(),
    }
}

func (sw *SlidingWindow) Allow() bool {
    sw.mu.Lock()
    defer sw.mu.Unlock()

    now := time.Now()
    elapsed := now.Sub(sw.windowStart)

    if elapsed >= sw.windowSize {
        periods := int(elapsed / sw.windowSize)
        if periods > 1 {
            sw.prevCount = 0
        } else {
            sw.prevCount = sw.currCount
        }
        sw.currCount = 0
        sw.windowStart = sw.windowStart.Add(time.Duration(periods) * sw.windowSize)
        elapsed = now.Sub(sw.windowStart)
    }

    fraction := float64(elapsed) / float64(sw.windowSize)
    estimated := float64(sw.prevCount)*(1-fraction) + float64(sw.currCount)

    if int(estimated) >= sw.limit {
        return false
    }

    sw.currCount++
    return true
}

This gives you a much tighter ceiling. A client that exhausts their quota in the first half of a window will not get a free reset at the boundary — the interpolated calculation accounts for recent history.

Wiring It Into HTTP Middleware

Both implementations expose the same Allow() bool interface, so you can drop either one behind a shared middleware. Here is a per-IP middleware for the standard net/http package:

package main

import (
    "net/http"
    "sync"
    "time"
)

type clientLimiters struct {
    mu       sync.Mutex
    limiters map[string]*SlidingWindow
}

func newClientLimiters() *clientLimiters {
    return &clientLimiters{limiters: make(map[string]*SlidingWindow)}
}

func (cl *clientLimiters) get(ip string) *SlidingWindow {
    cl.mu.Lock()
    defer cl.mu.Unlock()
    if _, ok := cl.limiters[ip]; !ok {
        cl.limiters[ip] = NewSlidingWindow(100, time.Minute)
    }
    return cl.limiters[ip]
}

func RateLimitMiddleware(cl *clientLimiters, next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        ip := r.RemoteAddr
        if !cl.get(ip).Allow() {
            w.Header().Set("Retry-After", "60")
            http.Error(w, "rate limit exceeded", http.StatusTooManyRequests)
            return
        }
        next.ServeHTTP(w, r)
    })
}

func main() {
    cl := newClientLimiters()
    mux := http.NewServeMux()
    mux.HandleFunc("/api/data", func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("ok"))
    })
    http.ListenAndServe(":8080", RateLimitMiddleware(cl, mux))
}

One production note: r.RemoteAddr includes the port. Behind a reverse proxy you will want X-Forwarded-For or X-Real-IP, but validate those headers carefully — a client that controls them can bypass per-IP limiting trivially. Trust only traffic from known proxy addresses.

Choosing Between the Two

Token bucket is the right default for general-purpose API rate limiting. It is simple to reason about, tolerates legitimate burst traffic, and is easy to tune. Most third-party integrations expect some burst allowance.

Sliding window is better when you are protecting sensitive endpoints where consistent throughput ceilings matter: authentication routes, password resets, payment processing. It is also easier to communicate to clients: "100 requests per minute, always" rather than "100 per minute with burst capacity."

For multi-instance deployments, move state out of process memory into Redis. The logic stays identical — you are replacing in-process maps with Redis atomic operations (INCR + EXPIRE for fixed windows, or Lua scripts for sliding window accuracy).

Additional hardening to layer on once the basics are working: per-endpoint limits on top of per-client limits, logging of rejected requests with client ID and reason (rejected requests are often the earliest signal of a brute-force or scraping attempt), and alerting when rejection rates spike.

The security hardening checklists we publish include rate limiting baselines for common web stacks — a useful starting point when configuring limits for production.

The Takeaway

Token bucket and sliding window both work. Token bucket is simpler and more permissive; sliding window is stricter and harder to game. The implementations above run in-process with no external dependencies and are straightforward to test with a simple ticker loop.

The bigger trap is skipping rate limiting entirely and bolting it on after an incident. The credential stuffing or scraping event will eventually happen — having limits in place before that day costs a few hours now and saves a lot more later.

I run AYI NEDJIMI Consultants, a cybersecurity consulting firm. We publish free security hardening checklists — PDF and Excel.