How to Map Untrusted Inputs in an AI Assistant

An order-status assistant can be limited to reading data and still show the wrong customer's information. Before adding another instruction to its prompt, draw the path from the user's message to the data it can retrieve.

This tutorial uses a fictional assistant. It is a design exercise, not a report of a tested production system. The text and diagrams were generated with AI.

1. Write down the assistant's job

For this example, the assistant answers a signed-in customer's questions about that customer's orders. It can request an order status through one tool. It cannot issue refunds, change orders, send email, or browse the web.

That sentence gives us a scope to review. If the implementation includes another tool, a shared document, or a memory store, add it to the map rather than assuming it is covered.

Customer message
       |
       v
AI assistant proposes an order lookup
       |
       v
Server checks the authenticated user's access
       |
       v
Order API returns permitted fields
       |
       v
Assistant answers the same customer

The server check is a proposed requirement in this example. Drawing it does not demonstrate that an application implements it.

2. Identify who controls each input

An input's origin helps determine how the application should use it. A customer message, a tool result, and an application-owned policy do not have the same role.

OWASP distinguishes direct prompt injection through user input from indirect injection through external content. Retrieved text should be handled as data; its wording cannot grant an assistant new authority. OWASP prompt injection overview.

3. Separate a valid ID from an authorized lookup

Suppose a signed-in customer supplies an order ID with the expected format. That tells the application something about the input's shape. It does not establish that the customer may access that order.

For this example, the tool handler needs to make an access decision using authenticated server-side context before returning order data. A user-supplied account name or a model's assertion of permission is insufficient evidence.

This is an ordinary application-security concern even when an LLM is involved. OWASP describes broken object-level authorization as a route to unauthorized access to another user's objects. A read operation can disclose information even when it cannot change a record. OWASP API1: Broken Object Level Authorization.

4. Turn the map into review questions

For each boundary, record five things:

1. Source: Who can supply or change this input?
2. Reach: Which data or action can it influence?
3. Enforcement: Where is permission checked outside the model?
4. Evidence: What observation would show that the check actually ran?
5. Gap: What remains unknown or needs testing?

A useful first record might read:

Order lookup: The ID can originate in customer chat. The server must check the signed-in user's permission for the requested order. The implementation and rejection behavior have not been inspected. Next step: review the handler and test access with authorized synthetic accounts.

The phrase "has not been inspected" preserves an important distinction: a missing description is a question to investigate, not proof that a control is absent.

OWASP's prevention guidance recommends tool-call validation, limited permissions, and monitoring alongside prompt-level measures. Clear formatting can help communicate intended roles, but it does not establish an authorization boundary. OWASP prevention guidance.

5. Prepare checks before making a safety claim

These are proposed checks, not executed results:

Run applicable checks only in a system you are authorized to assess. Keep the actual inputs, observed outputs, and relevant server evidence. A model saying "access denied" is not sufficient evidence that no protected data was retrieved behind the scenes.

The result of this exercise is a boundary inventory and a list of checks. It is not a penetration test, a security certification, or proof that prompt injection has been prevented.

Your next step

Choose one tool in your own authorized project. Record its source, reach, enforcement, evidence, and gaps. Then use that record to decide which implementation check should come next.