Finding the Bugs I Built an AI Agent to Prevent: Building Deterministic Guardrails with LangGraph & Python

``
A real debugging log: fabrication bugs, deployment gaps, and false test passes found and closed across 6 rounds of adversarial testing.

System prompts are a leash, not a lobotomy. A system prompt can be ignored, misread, or drift under pressure. Deterministic code cannot.

When I built Ahmad-AI — a local AI agent using LangGraph, Python, and Ollama with access to live SQL databases and Islamic knowledge bases — I established one non-negotiable design rule:

It must never state a specific citation or data claim unless that claim is independently verified. Verified, or it says nothing.

Because general-purpose LLMs fabricate citations confidently and often, "confidently wrong" is worse than "honestly uncertain." I spent weeks trying to prove the system would break its own rule anyway.

It did. Repeatedly. Here is what I learned across six rounds of adversarial testing against a live agent.

Round 1: The First Adversarial Pass

I ran a set of test questions against the live agent — empty database queries, Quran & Hadith citation requests, and ambiguous sorting queries.

Two critical bugs immediately surfaced:

The Ghost Row Bug:

When querying a table with no matching rows, the agent correctly output: "The query did not return any results." In the exact same sentence, it invented two fake item IDs with fake stock levels.

The Regex Mismatch:

When asked about usury (riba), the agent cited verse 5:51 (a real Quran verse, but entirely about interfaith alliances). My citation guard failed to fire because its regex expected Surah [name] X:Y, while the model wrote (Quran X:Y). A safety mechanism that doesn't match the shape of the output is functionally non-existent.

The Fix Philosophy: Code, Not More Instructions

My first instinct was to strengthen the system prompt. I resisted. Instead of writing tighter rules for the LLM, I built a deterministic backstop in Python that overrides the model’s prose when known failure patterns appear in state:

`python
python:
def apply_tool_result_guards(state, response):
"""
Deterministic backstop: if the last DB query came back empty or hard-errored,
override entirely with a fixed, honest message regardless of what the model wrote.
"""
if is_no_data(last_tool_message):
return "No matching data was found for that query. The data may not exist yet."

ALLOWED_CITATIONS = {"20:114"} # Pre-verified whitelist

def enforce_citation_guard(text):
for match in CITATION_PATTERN.finditer(text):
if not any(allowed in match.group(0) for allowed in ALLOWED_CITATIONS):
return SAFE_ISLAMIC_FALLBACK # Blocked even if the citation is real
return text

`

Building an Evaluation Harness That Doesn't Lie

Manually retyping test prompts in a terminal doesn't scale. To keep the agent honest, I split automated testing into two distinct layers:

Unit Tests (22 tests): Direct execution of Python guard functions against synthetic inputs (fast, no live LLM required).

End-to-End Evaluation Harness: Drives the live LangGraph execution against Ollama and Postgres.

Treating fuzzy model outputs as binary pass/fail either hides real regressions behind noisy false failures or downgrades hard security checks to "close enough."

Key Takeaways for AI Engineers

A whitelist beats a correctness checker: Verifying "is this citation accurate?" is hard. Checking "is this citation on the pre-approved whitelist?" is simple and reliable.

Print a Build Stamp at Startup: A version stamp printed at agent startup costs nothing and prevents testing against stale code.

Distinguish Transient from Permanent Errors: Retrying a connection drop makes sense. Retrying a bad SQL column name is just a slower way to fail.

Don't ask LLMs to do arithmetic or row matching: Stating query answers explicitly in tool outputs prevents models from misreading JSON rows.

Read the Full Engineering Log
This architecture is live, tested against a 22-unit-test suite and a 5/5 E2E hard check harness.

For the complete technical breakdown — including deployment gap fixes, bounded execution limits (MAX_TOOL_ROUNDS), and verbatim adversarial outputs:

📖 Read the full interactive log on my portfolio:

👉 Engineering Log 01: Guardrails & Hallucination Prevention

Muhammad Ahmad is an AI Agent Engineer and Digital Supply Chain Architect (PMP®, RMP®) with 15+ years of operational leadership experience.