API traffic monitoring: what to track, and how to set it up for a Node API

A customer emails: "your API has been timing out for an hour." Your uptime check is green, the error tracker is quiet and CPU looks fine. Somewhere in the last hour of traffic is the answer, and nothing you run kept it. API traffic monitoring closes that gap: record every request your API serves (who called which endpoint, with what payload, how long it took and what came back) and watch that traffic for errors, slow endpoints, breaking changes and abuse.

This guide covers what to track, how to capture it in a Node API with working code, and the queries that turn raw traffic into answers.

What is API traffic monitoring?

API traffic monitoring is recording the requests an API serves and watching that traffic for change. It works at the level your customers experience: one call, one endpoint, one consumer. The API traffic monitoring overview covers the category; this post is the hands-on version.

It overlaps with tools you may already run, but none of them keep the request itself:

What to track in API traffic

Six signals cover most of what goes wrong with an API in production.

Group everything by route pattern (/orders/:id), never the raw URL. Per-URL numbers are noise: every order ID becomes its own endpoint, and no percentile means anything.

How to capture API traffic in Node

Start with one structured record per request. This Express middleware uses Pino and records the fields every signal above needs:

import crypto from 'node:crypto';
import express from 'express';
import pino from 'pino';

const log = pino();
const app = express();

app.use(express.json());

app.use((req, res, next) => {
  const start = process.hrtime.bigint();

  res.on('finish', () => {
    log.info({
      type: 'api_request',
      method: req.method,
      // The pattern, not the raw URL: /orders/:id, not /orders/8f21a
      route: req.route ? req.baseUrl + req.route.path : 'unmatched',
      status: res.statusCode,
      durationMs: Math.round(Number(process.hrtime.bigint() - start) / 1e6),
      consumer: consumerOf(req),
      ip: req.ip,
      userAgent: req.get('user-agent') ?? null,
    });
  });

  next();
});

// A stable, non-reversible ID for the caller. Never log the key itself.
function consumerOf(req) {
  const key = req.get('x-api-key');
  if (!key) return 'anonymous';
  return crypto.createHash('sha256').update(key).digest('hex').slice(0, 12);
}

Three details matter:

• Mount it before your routes, so requests that never reach a handler (unmatched 404s, failed auth) are recorded too. Those are often the interesting ones.
• finish does not fire when the client disconnects first. Listen for close as well if you want to count abandoned requests, and record them with a status you can filter on.
• Behind a proxy or load balancer, set app.set('trust proxy', 1) (or the right hop count) so req.ip is the caller, not your load balancer.

Detecting response shape changes

Error rates stay flat when a deploy renames a field, but every client reading that field breaks. A cheap way to catch it is to record the shape of each JSON response (the keys and value types, not the values) and compare it per route:

const lastShape = new Map();

// { id: 7, tags: ['a'] } becomes { id: 'number', tags: ['string'] }
function shape(value) {
  if (Array.isArray(value)) return [value.length ? shape(value[0]) : 'empty'];
  if (value && typeof value === 'object') {
    return Object.fromEntries(Object.keys(value).sort().map((key) => [key, shape(value[key])]));
  }
  return value === null ? 'null' : typeof value;
}

export function watchShapes(log) {
  return (req, res, next) => {
    const json = res.json.bind(res);
    res.json = (body) => {
      const route = req.route ? req.baseUrl + req.route.path : 'unmatched';
      const key = `${req.method} ${route} ${res.statusCode}`;
      const current = JSON.stringify(shape(body));
      const previous = lastShape.get(key);
      if (previous && previous !== current) {
        log.warn({ type: 'response_shape_changed', key, previous, current });
      }
      lastShape.set(key, current);
      return json(body);
    };
    next();
  };
}

Mount it with app.use(watchShapes(log)) before your routes. The map is per process and forgets on restart, which is fine for a first version. Expect noise from optional fields and empty arrays: in practice you alert when a key that was present in most responses disappears, not on every difference.

Watching for abuse patterns

You do not need a security team to spot the common patterns, just the fields above:

• Enumeration: one consumer requesting many distinct IDs on the same route in a short window (/users/1, /users/2, /users/3...).
• Credential stuffing: login failures spread across many accounts from a small set of IPs.
• Scanning: a burst of 404s on paths you never served, such as /.env or /wp-admin.
• Missing or odd user agents on endpoints your real clients always call with one.

Turning traffic into answers

Ship the records somewhere you can query: a log platform, ClickHouse, or Postgres for a modest API. With the records in a Postgres table, one query gives you the per-endpoint view that matters:

SELECT route,
       count(*) AS requests,
       percentile_cont(0.95) WITHIN GROUP (ORDER BY duration_ms) AS p95_ms,
       avg((status >= 500)::int) AS error_rate,
       count(DISTINCT consumer) FILTER (WHERE status BETWEEN 400 AND 499) AS consumers_with_4xx
FROM api_requests
WHERE ts > now() - interval '15 minutes'
GROUP BY route
HAVING count(*) >= 50
ORDER BY p95_ms DESC;

The HAVING clause is the minimum volume: without it, two slow requests on a quiet route top the list. For reading the percentile columns correctly, see API latency percentiles; for which of these to page on, see API alerting best practices.

Two trade-offs to decide early:

• Metrics or records? Counters and histograms (Prometheus and friends) are cheap and great for alerting, but they cannot tell you which request failed or what it sent. Records can. Most teams want both: metrics to notice, records to explain.
• Payloads or not? Bodies are what make a request reproducible, and also where personal data lives. If you keep them, mask credentials and secret-looking fields in your process before anything is written, and set a retention period you would be comfortable explaining.

API traffic monitoring checklist

• One record per request, grouped by route pattern, mounted before your routes.
• A consumer ID on every record (hashed, never the raw key).
• p95 and error rate per route, with a minimum volume.
• Client errors counted by distinct consumers, not raw totals.
• Response shapes compared per route after each deploy.
• Credentials masked before anything leaves the process.

FAQ

What is the difference between API traffic monitoring and API monitoring?

"API monitoring" often means synthetic checks: a scheduled request that confirms an endpoint answers. API traffic monitoring watches the real requests your customers send, so it sees the failures a synthetic check never triggers.

Does API traffic monitoring slow down an API?

It should not. Record in-process, return immediately, and send the records in batches in the background, so no network call sits between your handler and your response.

Do I need an API gateway to monitor API traffic?

No. Middleware in your app sees everything a gateway sees, plus your route patterns and the parsed request, and there is no extra hop to run.

Doing this with ApiTraffic

Everything above works, and you will be maintaining it: the shape tracking, the queries, the retention, the masking. If you would rather not, ApiTraffic does API traffic monitoring for Node with one SDK (there are pages for Express, Fastify, NestJS and the other major frameworks), or for any stack through a proxy URL.

• Every request is recorded with its payload, route pattern, timing, firewall verdict and, with APM on, the queries it ran. Credentials are masked in your process by default.
• Search status_code >= 500 to see the failing requests themselves, or tag:userId == "1042" AND status_code >= 400 to check whether one customer is behind a spike (tags).
• Drift detection flags removed or retyped fields per endpoint, and each endpoint is alerted against its own baseline.

Start free and see every request your API handles: apitraffic.io