Dev.to WebDev 🛠 Dev 👁 0 📖 7 min read

Your API Takes 800ms. Here's How to Find Where the Time Goes

You call an API. It takes 800 milliseconds to respond. Your first instinct might be to optimize the database query, add Redis, or increase the number of application servers. But what if the database isn't the problem?

You call an API. It takes 800 milliseconds to respond.

Your first instinct might be to optimize the database query, add Redis, or increase the number of application servers.

But what if the database isn't the problem?

What if 200 milliseconds are spent establishing connections, another 300 milliseconds are lost waiting for a downstream service, and your application code only takes 20 milliseconds?

Before optimizing a system, you need to understand where the time actually goes.

Let's break down a real-world debugging workflow that you can use on almost any HTTP API.

1. Understand What Happens During an HTTP Request

A request travels through more components than we usually think about.

Here's a simplified request path:

              Your Browser
                   |
              DNS Lookup
                   |
            TCP Connection
                   |
             TLS Handshake
                   |
              CDN / Edge
                   |
             Load Balancer
                   |
              API Server
                   |
             Cache / Database
                   |
                Response
                   |
              Your Browser

Each step introduces some amount of latency.

An API response time isn't simply the time your backend spends executing code.

It's approximately:

Total Response Time
    = Connection Setup
    + Request Transfer
    + Proxy / Network Waiting
    + Backend Processing
    + Response Transfer

For browser applications, rendering and additional resource loading introduce more work after the response arrives.

This distinction becomes important when debugging production systems.

2. Measure the Request with curl

One of the simplest performance debugging tools is already installed on many developer machines: curl.

Run this command:

curl -o /dev/null -sS \
  -w '
DNS:         %{time_namelookup}s
TCP:         %{time_connect}s
TLS:         %{time_appconnect}s
Pretransfer: %{time_pretransfer}s
TTFB:        %{time_starttransfer}s
Total:       %{time_total}s
' \
  https://example.com/api/products

These metrics are cumulative timestamps measured from the beginning of the request.

For example, imagine getting these results:

DNS:         0.020s
TCP:         0.070s
TLS:         0.140s
Pretransfer: 0.145s
TTFB:        0.620s
Total:       0.800s

We can calculate the approximate time spent in each phase:

Phase Duration
DNS lookup 20 ms
TCP connection after DNS 50 ms
TLS handshake after TCP 70 ms
Additional pretransfer setup 5 ms
Waiting for first byte after setup 475 ms
Response download after first byte 180 ms
Total 800 ms

These are illustrative measurements, not benchmark results.

Notice something interesting.

The largest portion is the 475 ms between connection readiness and the first response byte.

Does that mean the database is slow?

Not necessarily.

That interval can include request transmission, reverse-proxy delays, backend execution, downstream service calls, and the return of the first response byte.

You need more information before deciding which component to optimize.

Also, these measurements describe this particular connection. Persistent HTTP connections, DNS caching, HTTP/2, and HTTP/3 can change which setup costs appear on subsequent requests.

3. Separate Network Latency from Backend Latency

Suppose your API passes through this infrastructure:

Client
  |
  v
CDN
  |
  v
Load Balancer
  |
  v
Application Server
  |
  +----> Redis
  |
  +----> PostgreSQL
  |
  +----> External API

Every component can contribute to the total latency.

A useful next step is to measure the time spent inside your application.

In a Node.js application using Express, you can start with simple middleware:

const { performance } = require("node:perf_hooks");

app.use((req, res, next) => {
  const start = performance.now();

  res.on("finish", () => {
    const duration = performance.now() - start;

    console.log(
      `${req.method} ${req.path} ` +
      `${res.statusCode} ${duration.toFixed(1)}ms`
    );
  });

  next();
});

This measures how long a request takes inside the Express processing pipeline until the response is finished from the application's perspective.

It doesn't tell you how long every downstream operation takes, but it's a useful starting point.

You can then instrument expensive operations individually:

async function measure(name, operation) {
  const start = performance.now();

  try {
    return await operation();
  } finally {
    const duration = performance.now() - start;

    console.log(
      `${name}: ${duration.toFixed(1)}ms`
    );
  }
}

Usage:

const products = await measure(
  "database.products",
  () => db.query(
    "SELECT id, name FROM products LIMIT 50"
  )
);

The db variable represents your application's database client.

In a production system, structured logs and distributed tracing are generally more useful than printing timings directly to the console.

But the principle stays the same:

Measure each meaningful operation rather than guessing which one is slow.

4. Understand Why More Servers Might Not Help

Imagine your API receives 1,000 requests per second.

Each request takes 200 milliseconds on average.

Using Little's Law:

Average Concurrent Requests
    = Requests per Second
    × Average Response Time

    = 1,000 × 0.2
    = 200

Under stable conditions, you have approximately 200 requests in progress.

Now suppose average response time increases to 800 milliseconds while throughput remains at 1,000 requests per second.

Average Concurrent Requests
    = 1,000 × 0.8
    = 800

You now have approximately four times as many requests in flight.

That can put additional pressure on:

  • Application memory
  • Connection pools
  • Database connections
  • Thread pools
  • Downstream services

The important insight is that latency and concurrency are connected.

Adding more application instances may improve throughput when the application servers are the bottleneck.

But if every request waits on the same overloaded database, increasing the number of application servers could make the database problem worse.

This is why scaling decisions should follow measurements, not assumptions.

5. Use Caching Where It Actually Helps

Caching is one of the most common system design techniques.

It's also one of the easiest to apply incorrectly.

Suppose your endpoint returns the same product metadata to thousands of users.

Without caching:

Request -> API -> Database -> Response
Request -> API -> Database -> Response
Request -> API -> Database -> Response

With a cache:

Request -> API -> Redis -> Response
                   |
              Cache Miss
                   |
                Database

If the cache hit rate is 95%, only approximately 5% of requests need to fetch the data from the database, assuming a basic cache-aside pattern.

For 10,000 requests per second:

Cache Hits:   9,500 requests/sec
Cache Misses:   500 requests/sec

This can dramatically reduce database read traffic.

However, caching introduces new questions:

How long should data remain cached?

A product description may tolerate slightly stale data. An inventory count or account balance may require stronger freshness guarantees.

What happens when the cache expires?

If thousands of requests try to rebuild the same cache entry simultaneously, you can create a cache stampede.

What happens when Redis becomes unavailable?

Your system may suddenly send the full request load to the database.

The correct approach is not simply "add Redis."

It's to understand the read patterns, freshness requirements, and failure modes before introducing caching.

6. Don't Optimize Average Latency Alone

Suppose your monitoring dashboard shows:

Average Response Time: 120ms

That sounds reasonable.

But the actual distribution might look like this:

p50:   80ms
p90:  220ms
p95:  450ms
p99: 1800ms

This means approximately 1% of requests take 1.8 seconds or longer.

For an application handling millions of requests per day, that can represent a significant number of slow user experiences.

Tail latency is especially important in distributed systems.

Consider a request that depends on several services:

             API Gateway
                  |
       +----------+----------+
       |          |          |
    User API   Order API  Product API
       |          |          |
       +----------+----------+
                  |
               Response

If those calls execute in parallel, the total time is largely constrained by the slowest required dependency, plus orchestration and network overhead.

If they execute sequentially, their latencies accumulate.

This means one intermittently slow service can affect the performance of an entire endpoint.

When investigating performance, track at least:

  • Request throughput
  • Error rate
  • p50, p95, and p99 latency
  • Database query duration
  • Connection pool saturation
  • Cache hit rate
  • Downstream service latency

Also monitor those metrics under realistic load, not just when the application is idle.

7. A Practical Debugging Checklist

The next time an API becomes slow, use this workflow.

Step 1: Reproduce the latency

Use curl to determine whether the time is spent in connection establishment, waiting for the first byte, or downloading the response.

Repeat the measurement and compare results across locations if necessary.

Step 2: Measure backend processing

Add request-level timing and identify expensive downstream operations.

If the system uses multiple services, add distributed tracing with something like OpenTelemetry.

Step 3: Inspect the database

Look for:

  • Slow queries
  • Missing or ineffective indexes
  • Lock contention
  • Connection pool exhaustion
  • Excessive database round trips

Use your database's query execution plans rather than assuming an index will solve every issue.

Step 4: Check service saturation

Monitor CPU, memory, active requests, queue depth, and connection pools.

Low CPU utilization doesn't necessarily mean a service has spare capacity. It may be waiting on I/O or a limited shared resource.

Step 5: Evaluate caching

Identify frequently repeated reads and determine whether they can tolerate stale data.

Measure the cache hit rate and have a plan for cache failures.

Step 6: Test failure scenarios

Ask what happens if:

  • The database becomes slow.
  • Redis becomes unavailable.
  • A downstream API times out.
  • Traffic suddenly increases fivefold.

Timeouts, bounded retries, circuit breakers, and load shedding may be needed to prevent a local failure from cascading.

8. The Bigger Lesson: System Design Is About Understanding Trade-offs

System design isn't about adding every possible component to an architecture diagram.

It's about understanding how a system behaves.

A load balancer doesn't automatically fix a slow database.

A cache doesn't automatically guarantee low latency.

Microservices don't automatically improve scalability.

And adding more servers doesn't always improve performance.

Each architectural decision should address a specific bottleneck, reliability requirement, or operational constraint.

A useful learning exercise is to follow a single HTTP request from DNS resolution through the application and database, then identify what happens when each component becomes slow or unavailable.

For those who prefer visual explanations, Designed to Scale includes animated system design lessons on request lifecycles, caching, load balancing, database scaling, message queues, and other distributed systems concepts.

The important part is to connect each concept to an observable behavior in a real application.

Final Thoughts

The next time someone says, "Our API is slow. We need Redis," ask a more useful question:

Where exactly are we spending the time?

Start with measurements.

Find the bottleneck.

Understand the trade-offs.

Then make the simplest change that addresses the actual problem.

That's the foundation of practical system design.

What has been the most surprising performance bottleneck you've encountered in a backend service?

📰 Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.