Write your agent as functions first, add a framework later

Plenty of agent tutorials open with a framework import. I'd start with ordinary functions that call a model, return structured data, and invoke tools in an order you can read top to bottom. A single model call is rarely the whole product. A compound AI system wires model calls, retrieval, tools, validation, and control logic into one pipeline, and quality depends on how those pieces fit together at least as much as on which model you picked.

Frameworks earn their keep when orchestration and state management solve problems your agent already has. Until you hit those problems, a dependency can hide the parts you still need to test, especially the data you feed the model.

Why plain functions first?

Functions give you a clear map of the workflow: what runs, in what order, and what gets passed to the next step. Branching stays explicit. You log inputs and outputs at boundaries you own. When something fails, the error surfaces in code you wrote and can read.

Structured output is the hinge. Models emit free text by default, which is fine for humans and fragile for programs. Ask for JSON that matches a schema when the next step is code: extract fields, classify a ticket, or choose a tool name and arguments. Enforced schemas beat loose JSON mode when you need the same fields every time. You still validate values and handle refusals or truncated responses.

Below is a sketch, not production code. It shows one planning step with a schema, then deterministic tool execution.

# Sketch: agent as functions + structured outputs (not production code)
from typing import Literal, Optional
from pydantic import BaseModel

class Plan(BaseModel):
    action: Literal["search_kb", "reply", "escalate"]
    query: Optional[str] = None
    reply_text: Optional[str] = None

def plan_next_step(user_message: str, kb_snippet: str) -> Plan:
    # Call your model API here with schema=Plan
    ...

def search_kb(query: str) -> str:
    return "..."

def run_turn(user_message: str) -> str:
    snippet = search_kb(user_message[:200])
    plan = plan_next_step(user_message, snippet)
    if plan.action == "search_kb" and plan.query:
        return search_kb(plan.query)
    if plan.action == "reply" and plan.reply_text:
        return plan.reply_text
    return "escalated"

That loop does not need a framework. Add retrieval, extra model passes, and checks as more functions. You are assembling the compound pipeline directly.

What signals mean you should reach for a framework?

Compare a framework to a small direct build of the same workflow. Where agent frameworks add value frames the decision around the problems your agent actually has.

Treat these as upgrade signals:

1. Debuggability under failure. When the agent misfires, can you inspect tool calls and application state without spelunking? If not, and failures are common, run traces and persisted step records may justify a dependency.

2. Interrupted work. Can you resume a long job without redoing a side effect (sending email, charging a card, writing a row)? If users or workflows need durable checkpoints, you are shopping for state management and replay semantics a framework may provide.

3. Branching and coordination. If control flow stops looking like a straight line and starts looking like parallel branches, human approvals, or handoffs between sub-agents, orchestration primitives replace ad hoc state machines you are tired of maintaining.

4. Retrieval iteration. Can you change how you fetch context and evaluate answers without fighting the framework's abstractions? If experimentation is slow, either simplify your stack or pick tooling that treats retrieval as pluggable.

Jerry Liu co-founded LlamaIndex and has argued that as agent patterns mature, stable abstractions leave more room to differentiate through production context. That describes where his company chose to invest. He still sees value in a framework layer when the application shape is open ended:

When there's a space where things are still being figured out, and then there's kind of a lot of flexibility in the end application, and that's where there's kind of value in a framework layer because the framework provides you know the core abstractions to make it easier to build certain things

Jerry Liu, LlamaIndex, Chain of Thought ep 61

On the same episode he ties that shift to context work:

Because the abstractions were solidifying, but the need for high-quality production context was still evolving.

Jerry Liu, LlamaIndex, Chain of Thought ep 61

What is each framework feature replacing?

Map features to pain you already feel. Here is how I would line up the hand-rolled version against what a framework might offer. Interrupt and resume: custom job IDs, idempotent tools, and your own tables of steps on one side; centralized run state, checkpoints, and replay on the other. Complex branching: nested conditionals and handwritten state machines versus graph or workflow primitives. Tool plumbing: per provider wrappers and argument parsing versus registries and calling conventions. Observability gaps: scattershot logging versus traces tied to tool calls.

If the hand-rolled column is still cheap for your team, the framework column may not pay for itself. A simple workflow may need little orchestration. A complex one may benefit from a framework. Neither choice removes the need to test the context supplied to the model.

The practical move is to compare both builds on the same workflow using the inspection questions from that debate: tool calls and state on failure, resume without repeated actions, whether branching got simpler, and whether retrieval changes are easy to evaluate.

For more on agent architecture, memory, and context as a moat, the best Chain of Thought episodes on AI agents collection is a useful set of alternate takes when you want voices beyond one framework founder.

How do you run the comparison fairly?

Build the smallest version of your workflow twice: once with functions and structured outputs, once with the framework you are considering. Same tools, same retrieval, same eval set. Run the four inspection questions on both paths. Let saved implementation time justify the dependency only if you also gain inspectability, resume behavior, or coordination you could not keep clear in plain code.

Do not adopt a framework because agent loops improved in the abstract. Adopt it when your codebase is already trying to become that framework.

Checklist

• Start with functions, explicit control flow, and schema-bound model outputs before adding orchestration dependencies.
• Treat the agent as a compound pipeline: model steps, retrieval, tools, validation, and glue code share ownership of quality.
• Reach for a framework when you need inspectable failures, resumable runs, heavy branching, or faster retrieval experiments than your current structure allows.
• Tie each framework feature to code you are maintaining today, not to a feature matrix on a landing page.
• Compare direct and framework implementations on the same workflow and the same eval data before you commit.

The longer explainer, with the episode clips, is on Chain of Thought. It draws on this episode.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.