How to build a security system for an AI platform in an era of new challenges and threats

Introduction

Hello everyone. Today's article will discuss my personal experience building a security architecture using examples from building my own platform. We'll explore hybrid and classic approaches, looking at real-world examples, challenges, and solutions.

When we started building the TozheBot platform , we had a rather unusual goal. We didn't just want to make a "platform for AI agents." We wanted to build a system in which classic security, edge isolation, and protection from next-generation threats work not as separate components, but as a single fabric . As a CTO, I've seen too many projects where encryption is in place, but data still leaks. Where multi-tenancy is advertised, but isolation is simply a filter in an SQL query. Where guardrails are added on top like a band-aid, rather than integrated into the architecture.

We took a different approach. Each platform security layer is designed with the existing layers in mind. Keys are rotated not just according to a schedule, but also with an eye on active sessions and pending migrations. Field encryption doesn't break data validation or knowledge base searches. Guardrails doesn't block legitimate requests, but it does detect LLM hallucinations and other threats. And all of this works in a multi-tenant environment, where a compromise by one tenant shouldn't affect others.

In this article, I want to show how theoretical principles are translated into working code, with real-world examples from our platform and the rationale behind each architectural decision.

So, let's begin the journey!

Classic security with a future-proof approach and key management

Selecting a technology

When choosing a cryptographic foundation, we had two options: use proven standards (such as AES-256-GCM) or take a risk and implement hybrid solutions with post-quantum resistance. The classical approach seems secure—until you consider the planning horizon. Data encrypted today can remain confidential for decades. The "store now, decrypt later" attack is not a theoretical threat: an attacker can intercept ciphertext today and decrypt it when a sufficiently powerful quantum computer becomes available. And with today's advances in this area, there's plenty to consider.

I've previously worked on hybrid algorithms, including implementing my own protocol, QuarkDash, which has demonstrated good results for chats. It solves this problem by combining the post-quantum security of Ring-LWE with the performance of symmetric ciphers ChaCha20 and Gimli. Benchmarks showed throughput of up to 2.8 GB/s and session establishment in less than 10 ms on commodity hardware, even in the TypeScript implementation, which fits well within the platform. This was critical for us: we couldn't sacrifice speed for security.

There is also a port for Go, which works even faster overall due to the language's features.

But it's not just about performance. QuarkDash's architecture allows for deriving keys not only through a classic exchange (as is done in chats), but also deterministically: from a master secret, salt, and version. This means we don't need to store the keys themselves in the database, only the metadata: salt, version, and KDF parameters. Compromising a long-term key doesn't reveal past sessions - a property known in cryptography as perfect forward secrecy.

In ours, KeyManager, this is implemented through a function deriveKeysthat adds four bytes with the version number to the salt:

// keyManager.ts: Return a keys with version-based salt
// It's just an example with basic parameters
private async deriveKeys(
  version: number,
  kdf: KdfParams,
): Promise<{ sessionKey: Buffer; macKey: Buffer }> {
  const versionBytes = Buffer.alloc(4)
  versionBytes.writeUInt32BE(Number(version), 0)
  const versionedSalt = new Uint8Array(Buffer.concat([this.salt, versionBytes]))

  const { sessionKey, macKey } = await QuarkDashPassphrase.deriveKeyForQuarkDash(
    this.options.masterSecret,
    versionedSalt,
    opts,
  )

  return {
    sessionKey: Buffer.from(sessionKey),
    macKey: Buffer.from(macKey),
  }
}

And here we come to the challenge of key rotation.

Key Rotation: When Security Meets Performance

Key rotation is a classic compliance requirement. But implementing it in a multi-tenant environment with millions of documents is a whole other headache. Rotating keys every 24 hours and simultaneously re-encrypting all data at once will result in downtime. If you don't re-encrypt, the old data will remain on the vulnerable key.

We solved this problem through lazy migration and background re-encryption. The key is rotated on a schedule, but the data is re-encrypted gradually, in batches of 100 documents, with a delay between batches. The process can be paused and resumed - the progress is saved in the keyring document.

The most interesting part here is the interaction with the platform's core. By default, our API automatically decrypts fields in afterRead hook. This means that during normal reading, we see plaintext, not ciphertext. But for decryption, we need the ciphertext - to determine the key version and whether the document requires migration. We solved this through the request context:

// A basic context example
const migrationReq = {
  user: undefined,
  context: { skipDecryption: true },
} as any

const result = await db.find({
  collection,
  limit: batchSize,
  where,
  sort: 'id',
  overrideAccess: true,
  depth: 0,
  req: migrationReq,
})

When skipDecryption set, decryptHook returns the value unchanged, and we see exactly what's in the database. When updating a document, the context is no longer passed, so encryptHook works as expected and re-encrypts the plaintext with the active key.

But that's not the whole headache.

Drift Protection: Why Rotation Lock Isn't Paranoia

In a multi-tenant environment, it's easy to end up with key versions accumulating faster than data can be migrated. For example, if each new user session triggers rotation, within a week we'll have 50 key versions, 49 of which contain data requiring migration.

To solve this problem, we have built in KeyManager three levels of protection:

• The first is a 5-minute cooldown for session-bound rotation.
• The second is a check for unfinished migrations: if there are pending background tasks, the rotation is blocked.
• Third, there's a limit on the number of versions (KEY_MAX_VERSION_DRIFT). If there are more than 50 versions, the system switches to read-only mode for rotation; migration must be completed first.

// Here is a simplified locking example
private checkRotationLock(): RotationLockCheck {
  if (this.explicitLock.locked) {
    return { locked: true, reason: this.explicitLock.reason ?? 'Explicitly locked' }
  }

  /* Pending migrations block rotation */
  const pending = this.getPendingJobs()
  if (pending.length > 0) {
    return {
      locked: true,
      reason: 'Pending re-encryption jobs in progress',
      details: { source: 'pending-jobs', jobs: pending.map(...) },
    }
  }

  /* Version drift - too many versions pile up = something is wrong */
  if (this.versionRecords.length > this.options.maxVersionDrift) {
    return {
      locked: true,
      reason: `Version drift exceeds KEY_MAX_VERSION_DRIFT (${this.options.maxVersionDrift})`,
    }
  }

  return { locked: false }
}

This isn't just a technical precaution. It's a compliance requirement. Regulators want to see that the system can explain why a key wasn't rotated and provide an audit trail. Ours checkRotationLock returns detailed information about the reason for the block, which is logged and accessible through admin endpoints.

This concludes encryption and moves on to another classic layer of security.

Loop Isolation and Zero Trust Architecture

Multitenancy: Why SQL Filters Are Not Isolation

The most common mistake in multi-tenant systems is relying on the developer to remember to add a filter WHERE tenant_id = ? to every request. I've seen this myself in dozens of projects: one missed filter, and one client's data leaks to another.

Our approach is contextual isolation. Every database operation is executed within the context of a tenant, and this context is passed down from the HTTP request to the tool invocation in the endpoint AI agents. agentRunner.ts explicitly extract the context tenantIdfrom the conversation and pass it to the tool execution context:

// The example of context
const tenantId = extractId(conversation?.tenant)

const toolCtx: ToolExecutionContext = {
  api,
  req,
  agentId: agent.id,
  conversationId: conversation.id,
  tenantId: tenantId!,
  userId: conversation.user,
  mcpServers,
  agentTools,
}

Every tool within the platform, whether a custom webhook create_lead or escalate_to_human a custom webhook, receives this context. This means that even if a tool attempts to access a collection without an explicit filter, the system can enforce isolation at the worker level. This isn't just defense-in-depth - it's the principle of least privilege by default.

Why don't we use a shared key pool for all tenants?

Returning briefly to the issue of encryption, another architectural choice: each tenant has its own set of encryption keys. This is more expensive from a management perspective (metadata needs to be stored for each tenant), but it's critical for compliance. If one tenant requires data deletion (GDPR or Federal Law No. 152-FZ), we can simply destroy its keys, making the data unreadable. If the keys were shared, we would have to re-encrypt the entire database.

This approach also reduces the damage radius of a compromise. If an attacker gains access to the keyring of one tenant, they won't be able to decrypt the data of others.

Naturally, this is far from all that's related to classic security. It also includes protection against CSRF, XSS, SQL, and other classic types of attacks, but we won't cover those today, as they've already become the foundation.

On that note, let's move on to more modern types of threats.

AI Agent Security: Where Classic Approaches Break Down

Prompt injections: why regular injections are just the beginning

The first thing that comes to mind when defending against prompt injections is regular expressions. "Ignore all previous instructions" is easy to detect. "Forget everything" is also easy. But modern attacks have long since moved beyond these obvious formulations. The OWASP LLM Top 10 clearly indicates that defense must be multi-layered: from isolating untrusted content to semantic filtering.

In addition to tenant isolation, we built a system with three complementary phases. The first phase is heuristics based on regular expressions. They operate instantly (0.1 ms) and catch approximately 30-40% of attacks. This isn't just a filter - it's a first line of defense that doesn't require loading an ML model and works even under resource-constrained conditions.

// A simple regex
const INJECTION_PATTERNS: Array<{ re: RegExp; score: number; label: string }> = [
  {
    re: /ignore\s+(all\s+)?(previous|above|prior)\s+instructions?/i,
    score: 0.95,
    label: 'ignore_previous',
  },
  { re: /jailbreak|dan\s+mode|developer\s+mode/i, score: 0.95, label: 'jailbreak' },
  {
    re: /(show|output|retry|say)\s+(your\s+)?(system\s+)?(prompt|instruction)/i,
    score: 0.95,
    label: 'leak_prompt_en',
  },
  // ... 100+ patterns
]

The next step is trained ML models. I'd previously experimented with this by creating the OGL-Mini library, so we used it as the basis for the future shield.

OGL-Mini: Why ONNX and Not an External API

The second phase is the OGL-Mini ONNX model I retrained. It was trained with TF-IDF 100,000 features and logistic regression. The latency on low-end hardware is 3-7 ms. So why did I choose a local model instead of an external API like OpenAI Moderation?

The answer is loop isolation. If every user request is sent to an external API for verification, we violate the "data never leaves the loop" principle. Furthermore, an external API adds 100-300 ms of latency due to the network request. This is critical for a chatbot. OGL-Mini runs on a CPU with a latency of less than 10 ms - that's 10-30 times faster than running to an external API.

And most importantly, this is unconditional control. We can further train the model using our own data, which we've observed during monitoring. We can add new attack classes without waiting for the external provider to update their model. And we can guarantee that the model is available even if the external API is unavailable.

Lazy Loading: Why We Don't Block App Startup

One of the most challenging architectural issues is when to load the ML model. If it's loaded at startup, the service will be unavailable for a few seconds. If it's not loaded at all, we lose the second phase of protection.

We chose lazy loading. On the first request, the model is loaded in the background, while the system continues processing regular expressions. Once the model is ready, it is included in the process. This ensures fault tolerance: even if the model fails to load, the underlying protection continues to operate.

// An example of lazy loading
function ensureGuardLoaded(): void {
  const status = getStatus()
  if (status.state === 'ready' || status.state === 'loading') return
  if (globalThis.__tozhebot_ogl_loading) return

  globalThis.__tozhebot_ogl_loading = (async () => {
    try {
      setStatus({ state: 'loading' })
      const mod: any = await import('hybrid-ai-guard').catch(() => null)
      // ... инициализация
    } catch (err) {
      setStatus({ state: 'failed', error: (err as Error).message })
    }
  })()
}

PII Masking: Why Tokens, Not Deletion

The third phase is personal data leakage protection. We don't just remove PII - we replace it with tokens. This is important because the model loses context if all names and phone numbers are removed from the conversation. Imagine a client writes, "My name is Elijah, my phone number is +7 999 123-45-67." If we remove this data, the model won't be able to create a lead.

We use stable tokens: the same name is always replaced with the same token within a session. This preserves the context for the model while protecting the data. After receiving a response from the LLM, we restore the original values:

export function maskPii(
  text: string,
  pii: { hasPII: boolean; entities?: PIIEntity[] },
): MaskResult {
  if (!pii.hasPII || !pii.entities?.length) {
    return { sanitized: text, tokenMap: new Map() }
  }

  const tokenMap = new Map<string, string>()
  const valueToToken = new Map<string, string>()
  const counters: Record<string, number> = {}

  const sorted = [...pii.entities].sort((a, b) => b.start - a.start)

  let out = text
  for (const ent of sorted) {
    const key = `${ent.type}:${ent.value}`
    let token = valueToToken.get(key)
    if (!token) {
      counters[ent.type] = (counters[ent.type] ?? 0) + 1
      token = `[${ent.type}_${counters[ent.type]}]`
      valueToToken.set(key, token)
    }
    tokenMap.set(token, ent.value)
    out = out.slice(0, ent.start) + token + out.slice(ent.end)
  }

  return { sanitized: out, tokenMap }
}

All of this is stored in encrypted form in dialogs. So even if the database is leaked, an attacker would only see something like this:

v5:3JIjNcvJf0XIqLKT348/eP83VZBD53U00bAsPnHryHx/wPaneeBWXNvv8ZbpaWY3KJfKjseVyyMN7Q==

Hallucination Detection: Why LLM Lies and What to Do About It

The most insidious problem with LLM is hallucinations. This is especially evident in earlier versions of the model (for example, GPT-6 Luna or GPT-5 Mini). The model might "think" it has called a tool and write something like, "I'm connecting you to an operator," in its response text. But in reality, no tool has been called. The user waits for an operator, and no one connects them.

We solve this in two ways. The first is by detecting JSON-like tool calls in the text. If the model writes {"name": "escalate_to_human"}, we notice it and execute the call manually:

// agentRunner.ts: Fallback to a JSON-like tool call
const fake = detectFakeToolCall(assistantText, availableToolNames)

if (fake) {
  yield { type: 'tool_call', toolName: fake.toolName }

  let toolResult = ''
  try {
    toolResult = await executeTool(fake.toolName, fake.args, toolCtx)
  } catch (err) {
    toolResult = `Tool execution error: ${(err as Error).message}`
  }

  yield { type: 'tool_result', toolName: fake.toolName, text: toolResult }
  // ...
}

The second is detecting escalation promises without actually calling the tool. If the model says "Connecting you to an operator" but doesn't call escalate_to_human, we force the escalation. This ensures the user isn't left in limbo.

Now that we've covered the basic layer of protection, let's move on to compliance.

Compliance as an architectural requirement, not bureaucracy

NIST AI RMF: Govern, Map, Measure, Manage

When we designed the system, we kept in mind the four functions of the NIST AI Risk Management Framework: Govern, Map, Measure, and Manage. For us, this isn't just a checklist, but an operational routine.

• Govern: This is our KeyManager encryption-based functionality. We can say at any time, "Rotation is blocked because a migration is in progress."
• Map: This EncryptedFieldsRegistry automatically collects a list of encrypted fields from the configuration of our database models. We don't need to maintain a separate registry - it's generated from the code.
• Measures are metrics such as how many documents have been migrated, how many key versions are active, and how many attacks on agents have been blocked. All of this is logged and accessible through admin endpoints.
• Manage is automatic actions: rotation, re-encryption, escalation upon detection of injections.

The platform itself offers analytics and an overview of blocked threats, analytics on the processing and storage of personal data with audit downloads, and much more.

EU AI Act: Why we're preparing in advance

Article 15 of the EU AI Act requires high-risk AI systems to be "accurate, reliable, and cybersecure" and to resist unauthorized modification. Our guardrails are precisely this protection that prevents manipulation of agent behavior. And logging all security events into a collection security-events is the basis for the reporting required by Article 73.

Audit: Why Logging Is More Than Just a "Good Practice"

In a multi-tenant environment, auditing is a must. We can't simply tell our clients, "We're protecting you." We must prove that our protection is working. Ours logSecurityEvent captures not only the type of event but also the context: which tenant, which user, which field, and what the attack probability is.

export async function logSecurityEvent(args: {
  api: Api
  req: ApiRequest
  tenantId?: string | number
  type: string
  severity: string
  input?: string
  reason?: string
  attackProbability?: number
  metadata?: Record<string, unknown>
}): Promise<void> {
  try {
    await args.api.create({
      collection: 'security-events',
      data: {
        type: args.type as any,
        severity: args.severity as any,
        tenant: args.tenantId as any,
        input: args.input,
        reason: args.reason,
        attackProbability: args.attackProbability,
        metadata: args.metadata,
      },
      overrideAccess: true,
      req: args.req,
    })
  } catch (err) {
    args.api.logger.warn({ err }, 'Failed to log security event')
  }
}

Now, let's move on to summing up.

Conclusion: Security as a single fabric

If you've read this far, you've probably noticed a pattern. Each component of our security system isn't isolated. KeyManager knows about background tasks because key rotation can break migration. encryptedField knows about skipDecryption because re-encryption requires raw ciphertext. guard.ts knows about PII masking because the model shouldn't see personal data. agentRunner knows about guardrails because escalation should be automatic. And so on.

When we talk about security principles, we're not talking about a list of abstract values. We're talking about concrete decisions that sometimes seem redundant - until they save the system. We adhere to several rules that inform every architectural choice we make.

We built a system in which security isn't a band-aid slapped on top. It's an architectural principle that permeates every layer: from cryptography to agent orchestration. And it's this coherence that makes the system resilient to threats we can't yet predict.

Security is not a state, but a process. Threats evolve, and we design systems with the expectation that tomorrow will bring something that doesn't exist today.

Quantum computers. The "store now, break later" attack is already possible today: an attacker can store our ciphertext and wait for a quantum computer to become powerful enough. QuarkDash with Ring-LWE makes this strategy pointless – data encrypted today will remain secure even after quantum machines become available. We're not waiting for NIST to fully standardize post-quantum algorithms – we're already using a hybrid scheme that combines classical and post-quantum security.

Automated attacks on LLM. Today, prompt injections are written by humans. Tomorrow, they will be generated by another LLM – thousands of variants per second, each with unique obfuscation. Our guardrails operate at the semantic level, not just patterns: OGL-Mini is trained on obfuscated examples , including base64, homoglyphs, and zero-width characters. Regular expressions complement the model, but do not replace it. We are prepared for the fact that tomorrow's attacks will become adaptive – and our architecture allows for retraining the model on new examples without rebuilding the entire system.

RAG knowledge base poisoning. If an attacker gains access to the knowledge base, they can inject instructions into it that LLM will interpret as actionable instructions. This is called indirect prompt injection, and the OWASP LLM Top 10 identifies it as a separate threat (LLM01). We already check RAG content for injection patterns before including it in context, and we plan to expand this check to the indexing stage. Data entering the RAG goes through the same process inspectInputas user input.

Supply chain attacks on ML models. ONNX models are binary files and can be spoofed. Our model modelDownloader implements the Trust-On-First-Use principle: the model hash is saved upon first loading and verified at each launch. If the hash doesn't match, the model is reloaded. We don't even trust our own models downloaded from a CDN; we verify their integrity at each launch. This protects against upstream repository compromise or MITM attacks during the loading phase.

Model inversion and data extraction. LLMs can "remember" training data and reproduce it on request. We don't train our models on user data; OGL-Mini is trained on public datasets. But even if we did, PII masking is implemented at the input: the model never sees real names and phone numbers. This isn't just data protection - it's protection against future attacks. If a reliable method for extracting training data via a chain of prompts were developed tomorrow, our models would no longer contain personal data.

Attacks on key management. If an attacker gains access to ENCRYPTION_MASTER_SECRET, they can decrypt all data. We minimize this risk through a versioned salt: compromising the master secret doesn't reveal past sessions unless the attacker also obtains the keyring document. However, we're already considering the next step: an HSM or KMS for storing the master secret. The architecture KeyManager allows for replacing the secret source without changing the key derivation logic - another example of cryptographic flexibility that pays off early.

We don't claim our system is invulnerable. That's not true. We claim that we must design for adaptability. Every layer is aware of the others. Every decision is documented. Every threat - even one that doesn't yet exist - leaves us room to maneuver.

This is holistic security: not a set of components, but a living system that learns faster than the adversary can attack.

Thank you for reading the article to the end.

I would be glad to receive your questions and additions!