Dev.to Security 🔐 Cybersecurity 👁 0 📖 6 min read

What a bank's security team asks before approving LLMs

Every regulated company we work with wanted to use LLMs, and every one of them hit the same wall before rolling them out company-wide. This post collects what their security and compliance teams asked for, and what we le

Every regulated company we work with wanted to use LLMs, and every one of them hit the same wall before rolling them out company-wide. This post collects what their security and compliance teams asked for, and what we learned when we put the same question to other practitioners on Reddit's r/LLMDevs.

The wall

Before any central control, the picture looked the same everywhere:

  • Each team had its own provider API keys, with no central view of them.
  • Nobody knew the monthly bill until it arrived.
  • There was no reliable record of who sent what to which model.
  • When security asked "what stops someone from pasting customer data or a secret into a prompt?", nobody had a good answer.

The use cases here are internal ones: assistants for support staff, document summarization, coding assistants for engineering teams. None of this is about LLMs making payment or credit decisions. But internal is exactly where the sensitive data question shows up, because people paste customer records into prompts.

Six requirements

What ended up being required before company-wide rollout:

  1. One internal endpoint. Applications call a single OpenAI-compatible address and never hold provider keys themselves.
  2. Limits and budgets. Limits per project, API key, user and model, plus monthly budgets in USD.
  3. Secret scanning and data masking before the request leaves the network, for things like national ID numbers and IBANs.
  4. Checks before the model call, so a blocked request costs nothing.
  5. A tamper-evident audit log that goes to the SIEM the security team already uses.
  6. Everything on their own infrastructure, not a SaaS proxy.

A data processing agreement with the provider does not replace masking. It covers what the provider promises, not what ends up in your own logs when someone pastes an IBAN into a prompt.

Who actually approves

One commenter argued that a bank's security team has no say, because compliance is set by the regulator. The regulator does set the rules. But someone inside the bank still has to decide whether a specific setup meets them, and that is the security and compliance team. The list above is what those teams asked for before they would sign off. It does not replace the regulation.

The audit log has to answer real questions

The first real audit question is usually "show me every request that was blocked last quarter, and who tried". Three things decide whether the log can answer it:

  • The policy version. If a rule changed mid-quarter, each decision has to say which version of the rule was active at that moment.
  • The SIEM as the primary store. If the gateway is the only place a record lives, its retention quietly becomes your audit retention.
  • Purges in the same chain. Deleting expired records is the one event that can make a record disappear for a legitimate reason. If the purge itself is not in the tamper-evident log, an auditor will find that gap first.

Should you keep prompt text?

The answer that came out of the thread was no, by default. Keeping prompt text for investigations means keeping exactly the material the investigation is about. The alternatives:

  • Record the verdict, not the matched text: which categories of data were found, which rule fired, under which policy version. That answers the regulator's real question ("what kind of data left the perimeter?") without holding the data.
  • Store a hash instead of the document. Most investigations start with someone handing over the document, so a hash is enough to confirm "was this exact file sent?".
  • If content is really needed, make it break-glass: capture for one project, for a fixed window, with an approval on record. The expiry should be enforced where the data is written, not in a settings screen, and reads of the captured content should be logged too.

The trade is stated plainly: you accept being unable to answer some questions, in exchange for not holding the material that would answer them. That limitation belongs in the audit documentation.

Coverage is a network problem

A gateway cannot see traffic that goes around it. A team with its own provider key, or someone using a chat app in the browser, never shows up in the logs, and the logs look clean.

The answer sits on the network side. Outbound access to the provider APIs is allowed only from the gateway's servers, so a personal key does not connect from a laptop. Browser AI apps are handled at the corporate proxy, either blocked or allowed on purpose. The gateway's part is to show which teams actually send traffic. A team with zero blocks for a whole quarter is either perfectly clean or not going through the gateway at all. And the timetable for revoking old provider keys is the real migration plan.

What would invalidate the approval

An approval is made under assumptions, and it should say what would break them. The most common one is the model itself: providers can change what sits behind the same model name. If the approval does not record which model version it covered, nobody notices when it quietly stops being true.

Residency is the other one. Choosing a provider's EU or local region does not change who operates the infrastructure, so the auditor's question becomes "who runs it?", not "where is the rack?". In Turkey the same question comes up under KVKK's rules on transferring personal data abroad. One option is to not treat it as all or nothing: mask everything before it leaves, and restrict the most sensitive projects to models running in-house.

How BaseLLM handles these today

What it does:

  • Audit log: every request is written to a hash-chained audit log that holds who, what and when, but no prompt text. It is never purged.
  • Decision records: policy, personal data and prompt injection decisions record the decision and the check that fired. They are kept for 90 days by default (configurable) and streamed to the SIEM, so the SIEM owns long-term retention.
  • Request logs: these hold project, key, model, tokens and latency only.
  • Masked samples: a personal data event can keep a short masked sample. This is a per-project setting (partial by default, full, or off). With it off, only the categories and counts remain.
  • External log tools: sending content to tools like Langfuse is per project and off by default. Even when it is on, the text goes out masked.
  • Model access: each project can be limited to specific models, so a sensitive project can be held to in-house models.

What it does not do yet:

  • Policy version: decisions do not yet record which version of the policy was active. This is on our list.
  • Purges: the retention job writes purges to the application log, not to the hash-chained audit log. Also on our list.
  • Hashes and break-glass: there is no document hash and no break-glass capture. We are not building them until a customer's audit needs them. A prompt's hash rarely matches a document pasted inside it, and break-glass would mean storing content that BaseLLM deliberately does not store today.

If you run LLMs in a bank, an insurer or another regulated company, we would like to hear what your auditors asked for. The 30-day evaluation is the easiest way to see BaseLLM on your own infrastructure.

Originally published on the BaseLLM blog.

📰 Read the original article on Dev.to Security

Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.