Dev.to AI 🤖 Ai 👁 0 📖 4 min read

GraphQL DataLoader for AI Gateway Resolvers

This post was created with AI assistance and reviewed for accuracy before publishing. GraphQL's N+1 problem is well known and usually discussed in terms of database rows. Putting a GraphQL gateway in front of AI service

This post was created with AI assistance and reviewed for accuracy before publishing.

GraphQL's N+1 problem is well known and usually discussed in terms of database rows. Putting a GraphQL gateway in front of AI services makes it considerably more expensive, because the thing being fetched N times is no longer a row. It is a model call, a policy check, or a tool registry lookup, each with real latency and sometimes a per-call price.

A query asking for twenty conversations and their summaries can become one query for the conversations and twenty separate summarisation calls. DataLoader is the standard fix, and the standard way to get it wrong is to share one across requests.

Why the resolver shape causes it

GraphQL resolvers run per field, per object. The engine has no way to know that twenty sibling resolvers are all going to call the same service:

const resolvers = {
  Conversation: {
    // Runs once per conversation in the result set.
    summary: (conv) => summariseService.get(conv.id),
  },
};

Ask for a list of twenty conversations and this fires twenty times. Each call is independent, so they run concurrently, which means the downstream service sees a burst of twenty rather than a steady stream. That is how a gateway trips its own provider's rate limits while looking idle in aggregate.

A loader collapses the burst into one call

DataLoader sits between the resolver and the service. It collects keys requested within a single tick of the event loop, then calls your batch function once with all of them:

import DataLoader from 'dataloader';

const summaryLoader = new DataLoader(async (ids: readonly string[]) => {
  const rows = await summariseService.getMany([...ids]);
  const byId = new Map(rows.map((r) => [r.id, r]));
  // Must return results in the same order as the keys.
  return ids.map((id) => byId.get(id) ?? null);
});

Two rules make the batch function correct. The array you return must be the same length as the keys array and in the same order, because DataLoader matches results to callers positionally. And a missing key must produce null or an Error in its slot rather than being omitted, otherwise every result after the gap resolves to the wrong object. That misalignment is a genuinely nasty bug: it returns real data, just attached to the wrong parent.

Create loaders per request, always

This is the mistake that matters most, and it is a security issue rather than a performance one.

// Wrong: one loader for the lifetime of the process.
const loader = new DataLoader(batchFn);
const server = new ApolloServer({ context: () => ({ loader }) });

// Right: a fresh set for every request.
const server = new ApolloServer({
  context: () => ({ loaders: createLoaders() }),
});

DataLoader caches. A module-scope loader therefore caches across users. The first request for conversation abc populates the cache, and every later request for abc gets that cached value regardless of who is asking, whether they still have access, or whether the underlying data has changed.

In a multi-tenant AI product that means one tenant's conversation summary served to another. The permission check in your resolver does not save you, because the cache hit happens inside the loader, below that check.

Per-request loaders give you batching within a request, which is where the N+1 lives, and no sharing across requests, which is where the risk lives. That is the correct trade.

Include the tenant in the cache key

Even within one request, keys that are not globally unique are dangerous. If a document id is only unique per tenant, key the loader on the pair:

const docLoader = new DataLoader(
  async (keys: readonly { tenantId: string; docId: string }[]) => { /* ... */ },
  { cacheKeyFn: (k) => `${k.tenantId}:${k.docId}` }
);

Object keys need an explicit cacheKeyFn, because the default cache uses identity comparison and two structurally identical objects are different keys. Without it you get no cache hits at all, which is a silent performance loss rather than a correctness one, but worth knowing.

Batching does not bound fan-out

DataLoader turns twenty calls into one. It does not stop a client asking for two thousand.

GraphQL lets the caller choose the shape of the query, and a nested query over an AI gateway can request an enormous amount of work in a few lines. Batching makes that cheaper without making it bounded.

Two limits are worth having:

Control Prevents
Query depth limit Deeply nested traversals that multiply resolver calls
Query cost analysis Wide queries that are shallow but request thousands of expensive fields

Cost analysis is the more useful of the two for AI backends, because the expensive fields are not necessarily the deep ones. Assign a weight to each field reflecting what it actually costs, sum it before execution, and reject queries over a threshold. A field that triggers a model call should carry a weight orders of magnitude above one that reads a column.

Measure calls, not milliseconds

The way to know any of this is working is to count downstream calls per GraphQL request. Latency alone will not tell you, because concurrent N+1 calls can look acceptable in wall-clock time while costing twenty times what they should.

context: () => {
  const counts = new Map<string, number>();
  return {
    loaders: createLoaders(counts),
    onFinish: () => logger.info({ downstreamCalls: Object.fromEntries(counts) }),
  };
}

Log that per request and the pathology is obvious: a request that fetched twenty conversations should show one summary call, not twenty. If you see the resolver count tracking the result count, a loader is either missing or being bypassed somewhere in that path.

That single log line is the whole feedback loop. Without it, an N+1 reintroduced by a new resolver is invisible until the provider bill arrives.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.