Dev.to AI 🤖 Ai Sat, 14 Nov 2026 19:46:29 +0000 👁 0 📖 8 min read

Context engineering — what replaced prompt engineering

"Prompt engineering" peaked as a term in 2023, got mocked through 2024, and by mid-2025 was being quietly replaced in serious engineering conversations with a broader label: context engineering. The shift is not just reb

"Prompt engineering" peaked as a term in 2023, got mocked through 2024, and by mid-2025 was being quietly replaced in serious engineering conversations with a broader label: context engineering. The shift is not just rebranding. The actual work of making an LLM reliably produce the right output turned out to be mostly about what information ends up in the context window — not the one magical system prompt line a prompt-engineering blog promised.

This article covers what context engineering actually is as a discipline in late 2026, the four strategies that cover most agent workloads, and the parts where the "new discipline" framing is still mostly marketing.

Where the term came from

The specific label took off in June 2025. Tobi Lütke (CEO of Shopify) framed it on X as "the art of providing all the context for the task to be plausibly solvable by the LLM." Andrej Karpathy amplified it roughly a week later, calling context engineering a "delicate art and science of filling the context window with the right information for the next step" and arguing explicitly that it should replace "prompt engineering" as the operative term for serious LLM work.

The point both were making: when a production LLM app fails, the fix is almost never a better prompt sentence. It is usually about what information was or was not placed in the context — the retrieved documents, the tool outputs, the conversation history, the system instructions, the examples, the user's intent. Prompt engineering treats the prompt as the unit of work; context engineering treats the entire context assembly as the unit of work.

What context actually contains

A prompt sent to an LLM is only one visible layer of a larger construct. The full context, in a typical production app, contains:

System instructions — the role, constraints, tone, and output format directives.
Retrieved information — chunks from a vector database (RAG), rows from a SQL query, API responses from a tool call.
Conversation history — the running record of previous user and assistant messages in a multi-turn session.
Working memory — intermediate conclusions the agent has reached in the current loop that are not yet in the user-visible transcript.
Tool definitions — the schemas of callable functions the LLM has access to.
Examples — few-shot demonstrations of the desired input-output pattern.
The current user turn — the actual message that triggered this call.

Each layer has a cost (tokens) and a benefit (better grounding). Context engineering is the discipline of deciding what goes in each layer, in what order, in what format, and when to drop or compress something to make room for something more important.

The four strategies for context (agent-focused)

For agent workloads specifically, four core strategies cover roughly 90% of production context engineering work:

Write

Store context OUTSIDE the LLM's context window so it survives across turns and sessions. Options:

Scratch pads — write intermediate reasoning to a text file or database, read it back on later turns.
Memory stores — write summaries of past conversations to a vector database, retrieve relevant ones on new turns.
Checkpoints — persist agent state (LangGraph's pattern) so a crashed workflow resumes without re-reading everything from scratch.

The failure mode this avoids: trying to keep all context in the LLM's active window and running out of tokens 20 turns into a conversation.

Select

Pull in only the context relevant to the current step, not everything available. Options:

Retrieval (RAG) — embed the current question, select the top-K relevant chunks.
Tool filtering — if an agent has 50 tools, show the LLM only the 5 that could plausibly apply to the current turn.
Memory filtering — of 1,000 past conversation summaries, select the 3 most similar to the current query.

The failure mode this avoids: dumping all possible context on every call, paying token cost without quality gain.

Compress

Reduce the token footprint of context already selected. Options:

Summarisation — replace a 10,000-token conversation history with a 500-token summary of what matters.
Extraction — pull the specific fields relevant from a long tool response rather than including the full response.
Prompt caching — mechanically distinct, but same spirit: pay less for the stable prefix. See the prompt caching article for the pricing side.

The failure mode this avoids: fitting but paying for giant context on every call, or spending latency processing information that was already extracted on a prior turn.

Isolate

Split a complex task across multiple LLM calls or sub-agents, each with a focused context, rather than one call with everything. Options:

Router + specialist pattern — a lightweight model routes the query to one of several specialists; each specialist has a context focused on its domain.
Multi-agent decomposition — one agent researches, another writes, another reviews. Each has only the context it needs.
Tool-use loops — the LLM asks for data, gets it, incorporates it, continues. Each sub-turn has focused context.

The failure mode this avoids: trying to fit "research sources + draft content + style guide + reviewer feedback" into one prompt and watching quality collapse under cognitive load.

When context engineering is overkill

Not every LLM workload needs this framing. For:

Pure one-shot classification — map an input to a label. Prompt + examples. No agent loop, no memory. Prompt engineering is fine as a term.
Short single-turn Q&A over a small corpus — fits in a prompt, no retrieval needed. Prompt engineering is fine.
Templated generation with fixed structure — invoice copy, standard emails. Prompt plus variable interpolation.

Context engineering becomes the right frame when any of these conditions hit:

The task is multi-turn with state that must persist across turns.
The information needed to answer is too large to fit entirely in the prompt.
The system has tools, and which tool to show the LLM depends on the turn.
The LLM is calling itself or another agent in a loop.
Quality is driven more by what-information-is-present than by how-the-question-is-worded.

If none of those apply, calling the work "context engineering" is a status upgrade on prompt engineering with no new content behind it.

The practitioner workflow

For a real agent system in late 2026, the context engineering loop looks like:

Enumerate what COULD go in the context. List every potential source: user turn, history, retrieval, tool outputs, memory, examples, system prompt.
Decide what SHOULD go in for this specific step. Not every source applies to every turn. A greeting message does not need retrieval; a technical question does.
Set a token budget per source. System: 500. Retrieval: 2000. History: 1000. Current turn: variable. Budgets prevent any one source from crowding out the others as the conversation grows.
Choose a compression strategy for sources near their budget cap. Full history → rolling summary + last 5 turns verbatim. Full tool output → extracted fields.
Order the context intentionally. System instructions first, then retrieved information, then history, then current user turn. Order affects what the model attends to.
Log what was actually sent and what was dropped. Instrument this before shipping. Debugging "why did the agent miss the last turn's context" is impossible without a log of what went in.
Evaluate at the context level, not just output level. Did the retrieved chunks actually contain the answer? Was the key tool definition included? These are context-side eval questions distinct from "was the final answer right."

A system that does these seven things reliably is doing context engineering whether anyone on the team uses that label.

Three failure modes to design around

1. Context window overflow. The conversation grows, context grows with it, eventually a single call exceeds the model's limit and the request fails outright. Fix: a hard budget on each source, with the "compress" strategies above kicking in when a budget is near.

2. The right information is present but buried. The chunk with the answer is at position 15 in a 20-chunk retrieval, and the model does not attend to it strongly because models over-weight the top and bottom of long contexts (the "lost in the middle" effect, well-documented since 2023). Fix: tighter retrieval (fewer, better chunks) and reranking so the most relevant lands at the top.

3. Tool definitions outnumber tool-use capacity. The LLM has 50 tool definitions in context but can only reasonably choose between a few at a time. Fix: route first (lightweight classifier chooses which 5 tools to expose), then let the model pick among those 5. See the agent frameworks article for how each framework handles routing.

Where the term is still mostly marketing

Three cases where "context engineering" is being used as rebrand with no new content:

A single prompt tweaked in a prompt-tuning tool. Still prompt engineering. The tool renamed itself but the practice did not change.
Adding RAG to a chatbot without any new discipline around what-goes-in-context-when. That is "we added RAG," not context engineering.
A conference talk titled "Context Engineering" with slides about system prompts. The term was used to make 2023 content feel current.

The honest test: if the work genuinely involves deciding what information from multiple sources ends up in each LLM call, in what order, in what compression, with what budget — it is context engineering as a discipline. If not, calling it that is naming not practice.

Bottom line

Context engineering is a real discipline, with four core strategies (write, select, compress, isolate) that cover most agent production work. The term is a reasonable upgrade from "prompt engineering" because the work genuinely shifted from "the sentence I write to the model" to "the whole bundle of information I assemble before each call."

Teams building anything beyond single-turn Q&A should adopt the framing — not for the label, but for the mental model shift. Every production LLM failure is some form of "the model did not have the right context at the right moment." Fixing that systematically is what the discipline is for.

If you're shipping a RAG or agent system to production, The RAG Production Handbook goes deep on the context-engineering decisions that actually move quality numbers — chunking, retrieval, evaluation, observability, and the full 7-field production log schema. Early-bird pricing for the first 20 buyers.