AI Agents for Backend Engineers: What Carries Over and What's New

If you write backend code, you're closer to building production AI agents than you think. Queues, retries, idempotency, logs: it all carries over. What's new is a component that makes decisions, gives different answers to the same input, and can be talked into things. Here's the translation layer.

An agent is an LLM with three upgrades

A plain LLM call can only talk. An agent can act, because it adds:

• Tools: functions it can call, such as querying a database or sending an email.
• Memory: a record of what has happened so far.
• A loop: it works step by step until the job is done. It thinks, acts, checks the result, and decides what's next. This pattern is called ReAct (reason + act).

Before you build one, ask whether a single, well-prompted LLM call would do. Summarising a paragraph is one call. Finding, fixing and testing a bug across 40 files is a job for an agent.

Tools: the model asks, your code decides

Function calling is the mechanism behind tools, and the most important thing to internalise is that the model never runs anything itself. You give it a menu of tools, each with a name, description and parameters. When it wants one, it replies with a structured request, and your code decides whether to run it. That's what keeps you in control.

Two habits make tools reliable:

1. Write descriptions like API docs, including when not to use the tool. "Fetch a customer's recent orders. Use for order status or refunds. Not for product search." That last sentence prevents a surprising number of wrong picks.
2. Validate arguments like any user input. Check the schema, check that referenced IDs exist, and rate-limit.

The Model Context Protocol (MCP), released by Anthropic in late 2024, lets you wrap a tool once and reuse it across any MCP-compatible agent. Think of it as USB-C for AI tools.

Memory is plumbing you build

LLMs remember nothing between calls; every bit of memory is something you store and feed back in. Short-term memory is the current conversation, limited by the context window. Long-term memory lives outside the model and is loaded only when relevant.

RAG (retrieval-augmented generation) is the common way to bring in outside knowledge: store documents in a vector database, retrieve the closest chunks, and put them in the prompt. Retrieve broadly, then re-rank and keep only the best few, and tell the model to answer using only that context so it doesn't guess.

Checkpointing means saving the agent's state after every step, so a restart resumes instead of starting over. It's the same reason you keep sessions in a database, not an in-memory dict.

Infrastructure: normal backend work, with two twists

Production agents are mostly the backend engineering you already do. The twists are that LLMs are non-deterministic and that their tool calls have real side effects. Four things keep you safe:

• Durable execution. Tools like Temporal save progress at every step, so a crash resumes where it left off.
• Sensible timeouts and retries. LLM calls can take a minute or more; back off politely on rate limits.
• Idempotency keys. A retried tool call must happen only once in the real world.
• Sandboxes. Run agent-written code in an isolated environment, never on your own server.

The idempotency key is the one engineers most often forget, and it's one line:

payments.refund(order_id="A123", amount=500, idempotency_key="refund:A123")
# The payment service sees the same key twice but refunds only once.

Cost matters too. Prompt caching lets providers reuse the repeated parts of your prompt; Anthropic reports savings of up to 90% on long prompts.

Observability: agents fail quietly

An agent's failures don't throw errors. You get a confident wrong answer, or a strange route to a right one, and a health check won't catch either. Tracing records every step as a span: what the model decided, which tool it called, what came back, the cost and the latency. Tools like Langfuse and LangSmith show this as a timeline.

Evaluation is testing for agents: plain assertions for exact things (right tool, right ID), and an LLM judge scoring fuzzy things against a rubric. Run evals in CI before every release, sample live traffic afterwards, and turn every production failure into a new test case.

Security: assume the model can be tricked

Agents act on text, and attackers can write text. The top risk is prompt injection: instructions hidden in a page, email or document the agent reads. You can't fully prevent it, so contain it:

• Least privilege, enforced in code, not in the prompt.
• Human approval for anything risky or irreversible: refunds, deletions, outgoing emails.
• Treat model output as untrusted. Never eval() it, and parameterise any SQL it touches.
• Keep secrets out of prompts. The model gets tools, never raw API keys.

The mantra: assume the model can be tricked, and make sure a tricked model can't do much damage. For example, an issue_refund() that raises NeedsHumanApproval above a set amount can't be pushed past that limit by a fooled model, because the code enforces the limit, not the prompt.

Go multi-agent last

Multi-agent systems are expensive and fragile. Anthropic's multi-agent research system used about 15 times more tokens than a normal chat, and a 2025 study found failure rates of 41% to 87% across multi-agent frameworks. What works best is one lead agent holding the full picture, with small helpers that do one job and report back a summary.

A learning path for a few weekends

1. Build a tiny agent with raw API calls and two tools, no framework.
2. Add RAG over a folder of your own notes.
3. Add tracing and 50 test cases, then break things on purpose.
4. Rebuild it in a framework: LangGraph, or the OpenAI or Claude Agent SDK.
5. Add durable execution and an approval step.
6. Expose a tool through MCP.

By step six you'll understand what frameworks do under the hood, which makes you much better at choosing and debugging them.

Originally published on TechBrosEbooks.

This article is adapted from the quick-start guide AI Agent Orchestration & Infrastructure for Software Engineers, which covers each idea in one short chapter, with orchestration patterns, multi-agent handoffs and A2A, an end-to-end support-ticket triage agent, and a pre-ship checklist.