AI Guardrails, Explained: Protecting Inputs and Outputs
A language model will do almost anything you ask it to. That flexibility is the reason the technology is useful, and it is also the reason it is risky to put in front of real users with real data. A model has no built-in
A language model will do almost anything you ask it to. That flexibility is the reason the technology is useful, and it is also the reason it is risky to put in front of real users with real data. A model has no built-in sense of which inputs are dangerous, which outputs would leak a secret, or which requests cross a policy line your organization cannot cross. Left to its own devices it will happily echo a social security number, repeat an injected instruction, or generate content your compliance team would never approve. Guardrails are the layer that supplies the judgment the model lacks.
A guardrail is a programmable check that sits between your application and the model and decides whether a given piece of text is allowed to pass, needs to be changed, or must be blocked. The most effective place to run these checks is the AI gateway guardrails layer that already intercepts every request, because a control that lives there applies consistently to every application without each team having to reimplement it. This article explains how guardrails work, where they run, and how to deploy them without breaking the experiences your users depend on.
Two sides of every conversation
The single most important idea in guardrails is that a model interaction has two sides, and each needs its own protection. There is the input, the text your application sends to the model, and the output, the text the model sends back. These carry different risks and call for different checks.
Input guardrails run before the model sees the request. Their job is to catch problems in what the user or application is about to send: a prompt-injection attempt, a block of personal data that should never leave your boundary, a request whose content violates policy. Catching these on the way in means the model never processes the dangerous material in the first place.
Output guardrails run after the model has produced a response but before that response reaches the user. Their job is to inspect what the model generated: a leaked credential, a piece of personal data the model reconstructed, unsafe code, content that fails a moderation standard. The model can produce problems even from a clean input, which is why inspecting the output is not optional.
When models call external tools, as agentic systems do through protocols like MCP, the same logic extends further. Checks can run pre-tool, before a tool is invoked, and post-tool, on what the tool returns, so that the guardrail boundary wraps not just the model but every action it takes.
Validate or mutate
When a guardrail finds something, it can respond in one of two modes, and the distinction shapes the user experience.
The first mode is validate. A validating guardrail makes a pass-or-fail judgment: the text is acceptable, or it is not. If it fails, the request is blocked. This is the right behavior for checks where any violation is unacceptable, such as a hard policy boundary.
The second mode is mutate. A mutating guardrail does not simply reject; it rewrites. The canonical example is PII redaction, where the guardrail finds personal data in the text and replaces it with a placeholder, allowing a sanitized version to continue. Mutation is powerful because it keeps the interaction alive instead of failing it, but a mutating guardrail can still block when the content is beyond repair. The choice between validate and mutate is really a choice about whether a problem can be fixed in place or must stop the request.
How strictly to enforce
Knowing what to check is only half the design. The other half is deciding what happens operationally when a check triggers, and this is where many deployments go wrong by being too aggressive too early. A mature guardrail system offers graded enforcement levels.
- Audit runs the guardrail and records what it would have done, without affecting the request. Nothing is blocked or changed; you simply collect data on how often the guardrail fires and whether its judgments look right.
- Enforce But Ignore On Error applies the guardrail's decision when the check runs cleanly, but if the guardrail itself errors, for example a dependency times out, the request is allowed through rather than failed. This protects you from a guardrail outage taking down your product.
- *Enforce * applies the decision strictly, including failing closed when the guardrail cannot run.
The recommended path is a staged rollout: begin in Audit to understand behavior and tune for false positives, move to Enforce But Ignore On Error once you trust the judgments but want resilience, and graduate to full Enforce for the checks that must never be bypassed. This progression lets you deploy protection without the painful surprise of a new guardrail blocking legitimate traffic on day one.
A streaming caveat worth knowing
There is one technical detail that trips up teams the first time they combine guardrails with streaming responses. When a model streams its answer token by token, with stream: true, the full output does not exist as a single object until the stream completes, so output guardrails are skipped for streaming responses. Input guardrails always run regardless, because the input is fully available before the model is called. If you need output checks to apply, you set stream: false so the complete response is available to inspect before it is returned. Understanding this trade-off between streaming responsiveness and output inspection is essential to designing a safe experience.
Applying guardrails: policies and headers
There are two complementary ways to decide which guardrails apply to a given request. The first is centralized: gateway policies that match on attributes of the request, such as the calling application, the target model, or the tool being invoked, and apply the appropriate guardrails automatically. This is how a security team enforces a baseline across the whole organization without depending on individual developers to opt in.
The second is per-request, using a header that specifies the guardrails for a single call. This gives application developers fine-grained control where they need it. A header-based configuration names the input and output guardrails to run:
{
"llm_input_guardrails": ["my-group/pii-redaction"],
"llm_output_guardrails": ["my-group/secrets-detection"]
}
Passed on an actual request, that looks like the following, where a scope header controls how many of the most recent messages the guardrail examines:
curl -X POST "{GATEWAY_BASE_URL}/chat/completions" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-H 'X-TFY-GUARDRAILS: {"llm_input_guardrails":["my-group/pii-redaction"]}' \
-H "X-TFY-GUARDRAILS-SCOPE: 10" \
-d '{"model":"openai/gpt-4o","messages":[{"role":"user","content":"Hello"}]}'
The scope value controls which messages are inspected: all of them, only the last, or a specific number of recent ones. A useful default is that system prompts are excluded from guardrail inspection, so your own carefully written instructions are not flagged as if they were untrusted user input.
Why the gateway is the right home for guardrails
A guardrail is only as valuable as the consistency with which it is applied. If each application team implements its own PII check, the organization ends up with a patchwork: slightly different rules, different coverage, and inevitable gaps where a team simply forgot. A new service that no one remembered to protect becomes the weak link, and an attacker needs only one. Running guardrails in the gateway solves this by making protection a property of the shared infrastructure rather than of each application's discipline. Every request that routes through the gateway inherits the baseline, and a security team can update a policy in one place and have it take effect everywhere at once.
This centralization also separates concerns cleanly. Application developers focus on building features, while the people responsible for safety and compliance own the guardrail policies. Neither has to depend on the other remembering to do the right thing on every request, which is exactly the kind of fragile coordination that security boundaries are meant to eliminate.
What guardrails can check
The checks themselves cover a broad range of risks. Content Moderation screens for unsafe or policy-violating material. PII and PHI Detection finds personal and health data. Prompt Injection detection looks for attempts to hijack the model's instructions; these three more computationally involved checks are often available in a hosted form. Secrets Detection catches credentials and keys. A Code Safety Linter inspects generated code, and a SQL Sanitizer guards against dangerous database statements. Regex Pattern Matching lets you enforce your own custom patterns. For agentic systems, policy engines such as Cedar, which can default to denying MCP tool calls unless explicitly allowed, and OPA provide fine-grained authorization over what tools an agent may use. Beyond these built-ins, a gateway typically integrates external guardrail providers and supports fully custom checks written in Python or built with frameworks like Guardrails AI, so you are never limited to a fixed menu.
Key takeaways
- Guardrails supply the judgment a model lacks, deciding whether text may pass, must be rewritten, or must be blocked.
- Every interaction has two sides: input guardrails run before the model and output guardrails run after, with pre-tool and post-tool checks extending the boundary to agentic tool calls.
- Validate mode blocks on failure; mutate mode rewrites, as in PII redaction, and can still block when content cannot be repaired.
- Stage enforcement from Audit to Enforce But Ignore On Error to Enforce so new guardrails never surprise production traffic.
- Output guardrails are skipped for streaming responses unless you set stream to false; input guardrails always run, and system prompts are excluded by default.
- Apply guardrails centrally through gateway policies or per request through headers, drawing on built-in, external, and custom checks.
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.