Getting Reliable JSON from LLMs with Pydantic
Asking an LLM to "reply in JSON" works most of the time. The rest of the time you get a friendly sentence before the JSON, a missing field, or a value your code doesn't expect, and your app crashes on json.loads. Pydant
Asking an LLM to "reply in JSON" works most of the time. The rest of the time you get a friendly sentence before the JSON, a missing field, or a value your code doesn't expect, and your app crashes on json.loads.
Pydantic turns that "most of the time" into something you can rely on. Here's the pattern I use.
1. Define the output as a model
Write down exactly what a valid answer looks like, including the allowed values:
from typing import Literal
from pydantic import BaseModel, Field, ValidationError
class TicketTriage(BaseModel):
category: Literal["billing", "bug", "feature_request", "other"]
priority: Literal["low", "medium", "high"]
summary: str = Field(max_length=200)
This one class is now your documentation, your validator and your type hints.
2. Show the model the schema
Pydantic can generate a JSON Schema from the class, so the prompt and the validation can never drift apart:
import json
schema = json.dumps(TicketTriage.model_json_schema(), indent=2)
prompt = f"""Classify this support ticket.
Reply with JSON only, matching this schema:
{schema}
Ticket:
{ticket_text}"""
If your provider supports a native JSON mode or structured outputs, turn it on as well. It cuts down on formatting mistakes, but you should still validate.
3. Validate, and retry once with the error
When validation fails, Pydantic's error message says exactly what's wrong. Sending it back to the model fixes most bad replies on the second attempt:
async def triage(ticket_text: str, llm) -> TicketTriage:
prompt = build_prompt(ticket_text)
for _ in range(2):
raw = await llm.complete(prompt)
try:
return TicketTriage.model_validate_json(raw)
except ValidationError as error:
prompt += f"\n\nYour last reply was invalid:\n{error}\nReply with corrected JSON only."
return TicketTriage(category="other", priority="medium", summary=ticket_text[:200])
The final fallback means the caller always gets a valid TicketTriage, never an exception from a bad generation.
4. Reuse the model in your API
With FastAPI, the same class can describe your endpoint's response, so the API contract and the LLM contract are one definition:
from fastapi import FastAPI
app = FastAPI()
@app.post("/tickets/triage", response_model=TicketTriage)
async def triage_ticket(ticket: TicketIn) -> TicketTriage:
return await triage(ticket.text, llm)
5. Keep the schema small
The more fields and nesting you ask for, the more ways a reply can be wrong. Ask only for what the next step of your code needs, use Literal for anything with a fixed set of values, and split large extractions into smaller calls.
Wrapping up
A Pydantic model, a schema in the prompt, one retry with the validation error, and a safe fallback. That small loop removes most of the fragility from LLM-powered features.
Do you use structured outputs in your projects? Share your approach below.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.