Multi-tenant from day one: keeping one customer's chatbot data out of another's

The first version of almost every B2B tool is single-tenant by accident. You build it for one customer, it works, a second customer arrives, and you add a customer_id column.

That column is now the only thing standing between your two customers' data. One forgotten WHERE clause and customer A's chatbot answers a question using customer B's internal pricing document. You will not get a second chance to explain that.

This is what I would set up on day one of a multi-tenant product, in the order the risk actually appears. It is written around a chatbot — vector search makes one of these problems unusually sharp — but most of it is the same for any SaaS.

1. Never pass the tenant id as an argument

The single highest-value change, and it is architectural rather than clever.

If getDocuments(tenantId, query) takes the tenant as a parameter, then every call site is a chance to pass the wrong one, or to forget it in a new function written at 1am. Humans do not reliably remember a rule 400 times.

Instead, resolve the tenant once, at the edge, from the authenticated session — never from a request body, a query string, or a header the client controls — and carry it in a context the data layer reads for itself:

// One place turns a session into a tenant. Nothing downstream takes it as an argument.
app.use(async (req, res, next) => {
  const session = await getSession(req);
  if (!session) return res.sendStatus(401);
  // The tenant comes from the session, never from req.body.tenantId.
  tenantContext.run({ tenantId: session.tenantId }, () => next());
});

// The data layer reads the context itself. There is no overload without a tenant.
function documents() {
  const { tenantId } = tenantContext.getStore() ?? {};
  if (!tenantId) throw new Error('No tenant in context — refusing to query');
  return db('documents').where({ tenant_id: tenantId });
}

That throw matters more than it looks. The failure mode you want for a missing tenant is a loud crash, not a query that quietly returns everything.

In Node this is AsyncLocalStorage. Most ecosystems have an equivalent. The point is not the mechanism — it is that the safe path is the only path, so a new developer cannot write the unsafe one without noticing.

2. Put the same rule in the database

Application-level filtering is one bug away from failing. A second, independent layer means a bug needs two mistakes to become a breach.

In Postgres that is row-level security:

ALTER TABLE documents ENABLE ROW LEVEL SECURITY;
ALTER TABLE documents FORCE ROW LEVEL SECURITY;   -- applies to the table owner too

CREATE POLICY tenant_isolation ON documents
  USING (tenant_id = current_setting('app.tenant_id')::uuid);

with the setting applied per transaction from the same context as above. FORCE is the line people miss: without it, the owner role bypasses the policy, and your application often connects as the owner.

This is defence in depth, not belt-and-braces theatre. The two layers fail for different reasons — one to a forgotten clause in code, the other to a misconfigured role — and it is unlikely you make both mistakes in the same hour.

3. Vector search is where isolation quietly breaks

Here is the chatbot-specific trap, and it is the one I would check first in anyone else's system.

A vector index does not have a WHERE clause in the way a SQL table does. Approximate nearest-neighbour search walks a graph and returns the k closest vectors it finds. If you then filter those results by tenant, two bad things happen:

• Correctness: you asked for 20 and got 20 from across all tenants; after filtering, you might have 3 for this tenant, or none. The bot answers badly for no visible reason.
• Safety: every piece of filtering logic between the index and the answer is a place to get it wrong.

Post-filtering is the wrong shape. The two shapes that work:

An index per tenant. Simplest to reason about, and the cross-tenant bug becomes structurally impossible — a query against tenant A's index cannot return tenant B's rows, whatever the code does. The cost is many small indexes, which is fine for hundreds of tenants and painful for a hundred thousand.

Pre-filtered search, where the tenant is a condition the index applies during the search, not after. pgvector does this naturally, because it is a Postgres query and row-level security already applies:

SELECT id, content, 1 - (embedding <=> $1) AS score
FROM   documents                      -- RLS already scopes this to the tenant
ORDER  BY embedding <=> $1
LIMIT  20;

Managed vector databases support metadata pre-filtering too — but read their documentation carefully and test it, because "filter" in some engines means post-filter, and the difference is exactly the one that matters here.

Write a test for this and never delete it. Two tenants, two documents, one query that would match the other tenant's document far better than its own:

test('a tenant never retrieves another tenant\'s content', async () => {
  await seed(tenantA, 'Our refund window is 30 days.');
  await seed(tenantB, 'Our refund window is 90 days.');

  const hits = await asTenant(tenantA, () => retrieve('how long is the refund window'));

  expect(hits).not.toHaveLength(0);                     // it still works
  expect(hits.every(h => h.tenantId === tenantA)).toBe(true);
  expect(hits.map(h => h.content).join(' ')).not.toMatch(/90 days/);
});

That last assertion is the one that catches a post-filter regression where the leak reaches the prompt but not the response object.

4. Secrets belong to one tenant each

Customers will give you API keys — their own model provider key, a WhatsApp token, an SMTP password. These need handling that plain columns do not give you:

• Encrypted at rest with authenticated encryption, AES-256-GCM or equivalent, so a database dump is not a credential dump.
• Decrypted only at the moment of use, never loaded into a general-purpose settings object that gets logged or serialised into an API response.
• Masked everywhere they are displayed. Show the last four characters. There is no screen that needs to show the whole key, including the owner's own settings page — they have it already.
• Scrubbed from errors. A provider's error object will happily include the request headers you sent.

5. Make the tenant visible in everything you operate

Isolation that you cannot observe is isolation you are only assuming.

• Logs carry the tenant id on every line. Debugging a cross-tenant issue without it is guesswork.
• Metrics and rate limits are per tenant. One customer's traffic spike should not degrade everyone, and a per-tenant limit is also your defence against a runaway loop in someone's integration costing you a fortune.
• Background jobs carry the tenant explicitly, because they run outside the request that had the context. This is the most common place the whole scheme leaks: a cron job with no session, written months later.
• Caches are keyed by tenant. A cache key of answer:${hash(question)} will serve tenant A's answer to tenant B, and it will do it intermittently, which makes it nearly impossible to reproduce.

That cache bug deserves emphasis. It is the one that gets shipped by careful teams, because the cache is added later as an optimisation, by someone who is thinking about latency rather than tenancy.

What it costs to do this on day one

Perhaps two days of work: the context plumbing, the RLS policies, the per-tenant index decision, the isolation test, the encrypted-secrets helper.

Retrofitting it onto a live product with real customers is weeks, because every query, job and cache written in the meantime has to be found and re-checked, and you cannot be sure you found them all. The second system is also the one where you are reading your own code wondering whether an untested path ever leaked.

The work is not hard. It is just much cheaper before the second customer than after the twentieth.

I build ChatNory, a multi-tenant AI chatbot that answers from each business's own content. The isolation test in section 3 exists because the post-filtering version looked completely fine until it did not. If you run vector search across tenants, go and check whether your engine pre-filters or post-filters — it is a five-minute check and it is worth doing today.