I Summed 400,000 Rows on Every Refresh

I had a little dashboard endpoint that summed a sales table by region. Nothing clever: filter the rows, add up the amounts, return a number. It felt instant in dev, so I stopped thinking about it. Then I left the page open on a second monitor. It auto-refreshes every few seconds, and when I tailed the logs I watched the same region total get recomputed over and over, each refresh redoing work whose answer hadn't moved since the last one. The underlying data changes maybe once an hour. I was aggregating 400,000 rows dozens of times a minute to produce a number I'd already produced.

The usual reflex is to reach for IMemoryCache and wrap the computation. That works, but it caches inside the handler, so you still pay for model binding, the handler invocation, and serializing the result every single time. What I actually wanted was for the whole response to come back without my code running at all. That's what output caching does, and it's been sitting in ASP.NET Core since .NET 7. I'd just never turned it on.

Three lines, basically

You register the services, add the middleware, and opt an endpoint in. Here's the shape of it:

builder.Services.AddOutputCache();   // services

var app = builder.Build();
app.UseOutputCache();                // middleware

app.MapGet("/sales-cached", (string region, SalesData data, HandlerCounter counter) =>
    Results.Ok(new { region, total = Summarize(region, data, counter) }))
   .CacheOutput(p => p.Expire(TimeSpan.FromMinutes(5)).SetVaryByQuery("region"));

That .CacheOutput(...) is the whole opt-in. I set a five-minute expiry because stale-by-five-minutes is fine for this dashboard, and I varied the cache entry by the region query value so ?region=west and ?region=east don't collide into one entry. A cached entry is keyed by the request path by default, so if a query value changes the answer, you have to say so explicitly.

The part I like is that the handler doesn't change. Same signature, same body, same dependencies injected. The caching lives in one chained call, not smeared through the logic.

Proving the handler actually stops running

I didn't want to take the docs' word for it, so I wired up a counter that increments every time the handler body executes, and stood up two endpoints doing identical work, one with .CacheOutput() and one without. Then I fired 50 requests at each (after one warm-up to keep JIT and connection setup out of the timing) and compared.

counter.Reset();
var uncachedMs = await HitRepeatedly("/sales-uncached");
var uncachedRuns = counter.Value;

counter.Reset();
var cachedMs = await HitRepeatedly("/sales-cached");
var cachedRuns = counter.Value;

The output, which was steady across a handful of runs:

Requests per endpoint : 50 (after one warm-up)
Uncached: handler ran  51 times,   112 ms total
Cached  : handler ran   1 times,    14 ms total
Speedup : 8.0x faster, 50 fewer handler executions

The timing bounced around a bit between runs (6.6x on a noisy one, 8x on a good one), and I wouldn't quote that number in a meeting, because it's a container and the absolute milliseconds depend on load. The number I trust is the execution count. 51 runs versus 1, every time, dead stable. The aggregation ran exactly once, populated the cache, and the other 50 responses came straight out of memory without touching my code. That's the thing output caching gives you that caching a value inside the handler can't: the handler, the binding, the serialization, none of it runs on a hit.

Where this quietly goes wrong

The failure mode that would scare me is caching something personalized under a shared key and serving one user's numbers to the next person who asks. Output caching won't do that to you by accident, since by default it skips requests that carry an Authorization header and responses that set cookies. But the moment you override the policy to cache an authenticated endpoint, you own that correctness yourself, and SetVaryByQuery won't save you if the thing that varies is the signed-in user. For anything per-user I'd rather not cache at the response boundary at all.

The other two are less dramatic. The cached number is as old as your expiry window, so a five-minute expiry means someone can see a five-minute-old total; that's fine for a dashboard and not fine for, say, an account balance. And the default store is in-memory per instance, so behind a load balancer each server keeps its own cache and your hit rate drops. There's an IOutputCacheStore seam for a shared store if you need it, but I'd reach for that only once I'd measured that per-instance caching wasn't enough.

My honest opinion after wiring this up: output caching is underused relative to how often people sprinkle IMemoryCache through their handlers. Caching the whole response for an endpoint that's genuinely read-mostly and the same for everyone is usually cleaner than caching fragments three layers deep, and it's a lot easier to reason about when something goes stale.

Here's the whole runnable sample, small enough to paste into a file and dotnet run:

using System.Diagnostics;

var builder = WebApplication.CreateBuilder(args);
builder.WebHost.UseUrls("http://127.0.0.1:5080");
builder.Services.AddOutputCache();
builder.Services.AddSingleton<SalesData>();
builder.Services.AddSingleton<HandlerCounter>();
builder.Logging.ClearProviders();

var app = builder.Build();
app.UseOutputCache();

static decimal Summarize(string region, SalesData data, HandlerCounter counter)
{
    counter.Increment();
    return data.TotalForRegion(region);
}

app.MapGet("/sales-uncached", (string region, SalesData data, HandlerCounter counter) =>
    Results.Ok(new { region, total = Summarize(region, data, counter) }));

app.MapGet("/sales-cached", (string region, SalesData data, HandlerCounter counter) =>
    Results.Ok(new { region, total = Summarize(region, data, counter) }))
   .CacheOutput(p => p.Expire(TimeSpan.FromMinutes(5)).SetVaryByQuery("region"));

await app.StartAsync();

var counter = app.Services.GetRequiredService<HandlerCounter>();
using var http = new HttpClient { BaseAddress = new Uri(app.Urls.First()) };
const int Requests = 50;
const string Query = "?region=west";

async Task<long> HitRepeatedly(string path)
{
    await http.GetAsync(path + Query); // warm-up
    var sw = Stopwatch.StartNew();
    for (var i = 0; i < Requests; i++)
        await http.GetAsync(path + Query);
    sw.Stop();
    return sw.ElapsedMilliseconds;
}

counter.Reset();
var uncachedMs = await HitRepeatedly("/sales-uncached");
var uncachedRuns = counter.Value;

counter.Reset();
var cachedMs = await HitRepeatedly("/sales-cached");
var cachedRuns = counter.Value;

Console.WriteLine($"Requests per endpoint : {Requests} (after one warm-up)");
Console.WriteLine($"Uncached: handler ran {uncachedRuns,3} times, {uncachedMs,5} ms total");
Console.WriteLine($"Cached  : handler ran {cachedRuns,3} times, {cachedMs,5} ms total");
Console.WriteLine($"Speedup : {(double)uncachedMs / Math.Max(cachedMs, 1):F1}x faster, " +
                  $"{uncachedRuns - cachedRuns} fewer handler executions");

await app.StopAsync();

sealed class SalesData
{
    private readonly (string Region, decimal Amount)[] _rows;
    public SalesData()
    {
        var regions = new[] { "west", "east", "north", "south" };
        var rng = new Random(42);
        _rows = new (string, decimal)[400_000];
        for (var i = 0; i < _rows.Length; i++)
            _rows[i] = (regions[i % regions.Length], (decimal)rng.Next(1, 10_000));
    }
    public decimal TotalForRegion(string region)
    {
        decimal total = 0;
        foreach (var row in _rows)
            if (row.Region == region) total += row.Amount;
        return total;
    }
}

sealed class HandlerCounter
{
    private int _count;
    public int Value => Volatile.Read(ref _count);
    public void Increment() => Interlocked.Increment(ref _count);
    public void Reset() => Volatile.Write(ref _count, 0);
}

The .csproj is just a Microsoft.NET.Sdk.Web project targeting net10.0, nothing special in it.

What's your rule of thumb for when an endpoint earns a .CacheOutput()? I keep going back and forth on whether to default read-only GETs to cached and opt out, or the reverse. Curious where other people draw that line.

— still caching answers nobody asked me to recompute