Your AI Agent Will Do Anything. That's the Problem (LLM Cost Guardrails)

If your product has an AI agent (an LLM with tools), every user prompt is spent on your bill — including the ones that have nothing to do with your product.

Picture a recipe app with a cooking assistant. A user types: "Write a Python script that prints the Fibonacci sequence, then run it 100 times and explain each result." A well-behaved general model will happily do it. It reasons, writes code, maybe calls a code tool, and returns thousands of tokens.

The user got a free coding assistant. You got the invoice. And nothing about that exchange helped anyone cook dinner.

This is the problem I want to highlight: most AI features are built to be capable, not to be scoped. An agent that will do anything is an agent anyone can use for anything, at your expense.

TL;DR: An LLM agent with no scope limits is a free, general-purpose AI on your bill. Put cheap checks first (rate limits, a topic gate), give the agent only the tools it needs, and cap tokens, steps and time on every run.

Why it matters

An unscoped agent costs you in three ways:

• Money. You pay per token, per tool call, per agent step. One off-topic prompt is cheap. Thousands a day, from users or scripts, are not.
• Abuse. Once people learn your free chatbot answers anything, it becomes a free proxy to a paid model. Bots will find it before your users do.
• Experience. Off-topic load slows the agent down for the people actually using your product.

This isn't a new or niche worry. OWASP lists it in its Top 10 for LLM applications as LLM10:2025 Unbounded Consumption, and names the cost side of it directly: "Denial of Wallet" — attackers exploiting pay-per-use pricing to run up a provider's bill (OWASP).

Why most apps miss it

Because the demo always works. When you build an agent, you test what it should do: the recipe question, the support ticket, the booking. You rarely test what it shouldn't do, because during development that costs nothing and nobody is trying to misuse it.

The second reason is visibility. A single request's cost is invisible in the UI. It only shows up at the end of the month, as one number on a provider invoice, with no clue which prompts caused it.

And the third: a system prompt that says "You are a cooking assistant" feels like a boundary. It isn't. It's a suggestion the model weighs against whatever the user asks next.

How to keep an agent inside its scope

No single trick does it. What works is layers, ordered so the cheapest checks run first and the expensive model runs last — and only for requests that belong in your product.

1. Quotas before anything else. Check the user's rate limit and daily budget before a single token is spent. This check is free, and it stops scripts and runaway loops cold.
2. Limit the input. Cap prompt length and attachment size. A 50-page paste into a cooking assistant is not a cooking question.
3. Put a cheap gate in front. Ask a small, fast model (or a simple classifier) one question: "Is this request about cooking?" Off-topic requests get a fixed, friendly reply — the big model never runs. The gate costs a fraction of the agent call it prevents.
4. Scope the system prompt too. Tell the agent what it is for and how to decline the rest. This is your second line, not your only one.
5. Only the tools it needs. A recipe agent needs recipe search and a shopping list. It does not need a general code runner or open web browsing. Every extra tool widens what "anything" means.
6. Hard budgets on every run. Cap output tokens, agent steps, tool calls and wall-clock time per request. The Fibonacci-100-times prompt hits the step limit and stops, instead of looping on your bill.
7. Watch it. Log tokens and cost per request and per user. Alert on spikes. Read the top refused prompts every week — they tell you what users actually want.

The budget part is a few lines of code, which is exactly why it's worth having:

MAX_STEPS, MAX_TOOL_CALLS, MAX_SECONDS = 8, 5, 30

for step in range(MAX_STEPS):
    if tool_calls > MAX_TOOL_CALLS or elapsed() > MAX_SECONDS:
        return "That's more than I can do in one go. Can you narrow it down?"
    result = agent.step(max_output_tokens=800)
    if result.done:
        return result.answer

Put together, a request flows like this:

Most off-topic and abusive traffic stops at the first two boxes, where it costs little or nothing.

Don't lock it down too hard

A scope gate that's too strict is its own bug. "What wine goes with this pasta?" is not strictly cooking, but a cooking assistant that refuses it feels broken.

A few rules keep the balance:

• Define scope by the user's goal, not by keywords. "Help me plan and cook meals" covers wine, budgets and grocery lists. "Only recipes" doesn't.
• Redirect, don't scold. "I'm your cooking assistant, so I can't help with code — but I can suggest a quick dinner" keeps the user in the product.
• Measure both errors. Track off-topic requests that got through and real requests that were wrongly refused. Tune the gate against both.

A checklist before you ship an agent

• ✅ Per-user rate limits and a daily token or cost cap
• ✅ Maximum input length
• ✅ A cheap scope check before the main model runs
• ✅ A system prompt that states the scope and how to decline
• ✅ Only the tools the product needs
• ✅ Limits on output tokens, agent steps, tool calls and time per request
• ✅ Cost logged per request and per user, with spike alerts
• ✅ A weekly look at refused and most expensive prompts

Capability gets the demo. Scope keeps the business alive. If you're shipping an AI feature, ask one question before launch: what stops someone from using it for something else, on my bill?

How are you handling this in your own products? I'd like to hear what's worked.

Sources

• OWASP Top 10 for LLM Applications: LLM10:2025 Unbounded Consumption