DSPy: stop hand-tuning prompts, write signatures and let a metric pick the prompt

Thursday AI Tool/Workflow. This week's tool: DSPy, the open-source Python framework from Stanford NLP for "programming, not prompting" language models. You describe a task as typed inputs and outputs, pick a module that decides how the model runs it, and then let an optimizer tune the prompt against a metric you write.

If you've shipped an LLM feature, you know the loop: tweak the prompt, eyeball five outputs, ship, then a model update or a new edge case breaks it and you start again. The prompt is a long string nobody wants to touch, and there's no number telling you whether the last edit helped.

DSPy turns that loop into code you can test. Here's exactly how to use it.

What DSPy gives you

From the official docs, the three pieces that matter:

• Signatures: declare the task as typed input and output fields (with Literal, Optional, lists, even dspy.Image) instead of writing a prompt.
• Modules: control how the signature runs. dspy.Predict is a direct call, dspy.ChainOfThought adds step-by-step reasoning, dspy.ReAct adds tools and a reasoning loop. Same signature, different strategy.
• Optimizers: give DSPy examples and a scoring function and it tunes the prompt (instructions and/or few-shot demos) for you. BootstrapFewShot, MIPROv2 and GEPA all ship in the package.

It's MIT-licensed, and DSPy 3.4.1 (October 2026) on PyPI requires Python 3.10 or newer.

1. Install and connect a model

pip install dspy

import dspy

lm = dspy.LM("openai/gpt-5-nano", api_key="YOUR_OPENAI_API_KEY")
dspy.configure(lm=lm)

print(lm("What is the capital of France?"))
# ['The capital of France is Paris.']

Models use a "provider/model" string. The docs list native routes for Anthropic (anthropic/...), Google AI Studio (gemini/...), Azure OpenAI, Vertex AI and local Ollama (ollama_chat/llama3.2:1b), plus any OpenAI-compatible server via api_base:

lm = dspy.LM(
    "openai/your-model-name",
    api_base="https://your-provider.example/v1",
    api_key="YOUR_PROVIDER_KEY",
)

dspy.configure(lm=...) sets the default for the whole process. Use with dspy.context(lm=other_lm): to override it for one block.

2. Your first signature

Here's a support ticket classifier. Notice there is no prompt anywhere:

from typing import Literal
import dspy

class TriageTicket(dspy.Signature):
    """Classify a customer support ticket for routing."""

    ticket: str = dspy.InputField()
    category: Literal["billing", "bug", "account", "feature_request"] = dspy.OutputField()
    urgency: Literal["low", "medium", "high"] = dspy.OutputField()

triage = dspy.ChainOfThought(TriageTicket)

pred = triage(ticket="I was charged twice for my subscription this month.")
print(pred.category, pred.urgency)   # e.g. billing high
print(pred.reasoning)                # ChainOfThought adds a reasoning field

The docstring becomes the instruction, the field names and types become the format, and the Literal types tell the model (and DSPy's parser) which values are allowed. For quick experiments you can also write signatures inline as strings, like dspy.Predict("question -> answer").

Want to see what was actually sent to the model?

dspy.inspect_history(n=1)

This prints the full system and user messages plus the raw response. Run it once; it's the fastest way to stop thinking of DSPy as magic.

3. Write a metric and get a baseline

Optimizers need to know what "better" means. A metric is a plain Python function that takes a labeled example and the program's prediction and returns a score (by convention 0.0 to 1.0).

# Labeled tickets: in real life, pull 50-200 from your helpdesk history
rows = [
    ("I was charged twice this month", "billing", "high"),
    ("App crashes every time I log in", "bug", "high"),
    ("Can you add a dark mode?", "feature_request", "low"),
    ("How do I change the email on my account?", "account", "low"),
    # ...
]

examples = [
    dspy.Example(ticket=t, category=c, urgency=u).with_inputs("ticket")
    for t, c, u in rows
]
trainset, devset = examples[: len(examples) // 2], examples[len(examples) // 2 :]

def triage_metric(example, pred, trace=None):
    category_ok = example.category == pred.category
    urgency_ok = example.urgency == pred.urgency
    if trace is not None:            # during bootstrapping: only keep perfect demos
        return category_ok and urgency_ok
    return (category_ok + urgency_ok) / 2

evaluate = dspy.Evaluate(devset=devset, metric=triage_metric, num_threads=4, display_progress=True)
baseline = evaluate(triage)
print(baseline.score)   # percentage, e.g. 50.0

.with_inputs("ticket") tells DSPy which fields go into the program; the rest (category, urgency) are held back as labels for the metric.

4. Optimize

The simplest optimizer, BootstrapFewShot, runs your program over the training set, keeps the traces your metric scores as correct, and puts them into the prompt as demonstrations:

optimizer = dspy.BootstrapFewShot(
    metric=triage_metric,
    max_bootstrapped_demos=4,
    max_labeled_demos=4,
)
optimized = optimizer.compile(triage, trainset=trainset)

print(evaluate(optimized).score)

For bigger gains, the docs now lead with GEPA, which rewrites the instructions using a reflection model. Its metric can return text feedback explaining why a prediction failed, and the reflection LM uses that feedback to write better instructions:

def triage_feedback(example, pred, trace=None, pred_name=None, pred_trace=None):
    problems = []
    if example.category != pred.category:
        problems.append(f"category should be {example.category}, got {pred.category}")
    if example.urgency != pred.urgency:
        problems.append(f"urgency should be {example.urgency}, got {pred.urgency}")
    score = 1.0 - len(problems) / 2
    return dspy.Prediction(score=score, feedback="; ".join(problems) or None)

gepa = dspy.GEPA(
    metric=triage_feedback,
    reflection_lm=dspy.LM("openai/gpt-5.4"),  # a stronger model to write instructions
    auto="light",                              # budget: "light", "medium" or "heavy"
    num_threads=2,
)
optimized = gepa.compile(triage, trainset=trainset, valset=devset)

Per the GEPA guide, trainset feeds the reflective prompt updates and valset selects which candidate compile returns. Keep a separate test set that neither step sees, and score the final program on that.

5. Save, load, ship

optimized.save("triage_v1.json")

# in your API service
triage = dspy.ChainOfThought(TriageTicket)
triage.load("triage_v1.json")

The JSON holds the program state: the signature instructions, the selected demos, and the DSPy and Python versions it was saved with. Commit it next to your code and review diffs like any other config.

6. Example workflow: re-tuning the ticket router for a cheaper model

This is an example workflow, not a benchmark. Say your helpdesk router runs on a large model and you want to move it to a smaller, cheaper one without losing accuracy.

1. Freeze the task. TriageTicket stays the same. Only the LM changes.
2. Build the eval set once. Export 200 resolved tickets with their final category and urgency. Split into train, val and a held-out test set.
3. Baseline both models. Run dspy.Evaluate on the test set with the large model and the small one (with dspy.context(lm=small_lm):).
4. Compile for the small model. Run GEPA with the small model as the program LM and the large model as reflection_lm.
5. Gate the release. Ship triage_small_v1.json only if its test score is at least the large model's baseline. Otherwise keep the old one.
6. Repeat on every model release. When a new model ships, rerun steps 3 to 5. The docs make exactly this point: you can rerun the optimizer against a new model and immediately see if it's worth swapping in.

The DSPy docs cite production examples of this pattern: Shopify moved a single-prompt GPT-5 task to a small Qwen model optimized with GEPA, reporting a result about 75x cheaper and about 2x more reliable; Dropbox used DSPy optimizers to move to a smaller model and double program accuracy.

Gotchas before you ship

• The cache will fool you. DSPy caches LM responses in memory and on disk by default, so calling the same program with the same input twice returns the first answer instantly. Great for cost, confusing when you're testing sampling. Use dspy.LM(..., cache=False) or dspy.configure_cache(enable_disk_cache=False, enable_memory_cache=False) when you need fresh calls.
• Your metric is the spec. The optimizer selects for whatever the metric rewards. A sloppy metric produces a confidently optimized wrong program. Write it like a unit test.
• Don't optimize on your test set. Train for proposals, val for selection, test for the number you report.
• Watch optimizer cost. Optimizers call the model many times. Start with auto="light", set num_threads low enough for your provider's rate limits, and turn on dspy.configure(lm=lm, track_usage=True) to see token usage per prediction.
• Read the prompt. After compiling, run dspy.inspect_history() or open the saved JSON. The optimized instructions are often long; make sure nothing in them leaks data or encodes a bias you don't want.
• Pin the version. DSPy moves fast (3.3.0 landed in August 2026, 3.4.0 in September, 3.4.1 in October). The saved JSON records the version; pin dspy== in production and re-check the migration notes when upgrading.

Prompts are a parameter, not a source file. Give them a metric and let the computer do the tuning.

References

• DSPy documentation: https://dspy.ai/
• DSPy on GitHub: https://github.com/stanfordnlp/dspy
• DSPy on PyPI: https://pypi.org/project/dspy/
• Setting up DSPy (install, LMs, providers): https://dspy.ai/learn/programming/language_models/
• Metrics and evaluation: https://dspy.ai/learn/evaluation/metrics/
• Prompt optimizing with GEPA: https://dspy.ai/learn/optimization/optimizers/
• Cache tutorial: https://dspy.ai/tutorials/cache/
• DSPy paper (arXiv 2310.03714): https://arxiv.org/abs/2310.03714
• GEPA paper (arXiv 2507.19457): https://arxiv.org/abs/2507.19457

How do you decide a prompt change is actually better today: an eval set, or vibes?