Prompt Chaining with Spring AI: Four Small Steps Beat One Big Prompt
Prompt Chaining with Spring AI: Four Small Steps Beat One Big Prompt I keep running into the same temptation when I build LLM features. I write one long prompt that asks the model to extract some data, clean it up, sor
Prompt Chaining with Spring AI: Four Small Steps Beat One Big Prompt
I keep running into the same temptation when I build LLM features. I write one long prompt that asks the model to extract some data, clean it up, sort it, and format it, all in one go. Sometimes it works. Often the result is almost right: a number is in the wrong unit, a row is missing, or the formatting drifts. Then I spend more time tuning the big prompt than the problem deserved.
Prompt chaining is the pattern I reach for when that happens. Instead of one prompt, you write a sequence of smaller prompts. Each step receives the output of the previous one and does one transformation. The steps are simple enough to test in isolation, and when something goes wrong you can see which step broke.
Anthropic describes prompt chaining as one of the basic workflows in its research on building effective agents. The sample project in the spring-ai-examples repository implements it with Spring AI, and in this article I walk through that sample: how the chain is built, what each step does, where the code is simpler than the README suggests, and how I would harden it before using it in a real service.
Where prompt chaining fits
The pattern is a fixed workflow. The developer decides the steps in advance, and the model does the work inside each step. That makes it predictable, cheap to reason about, and easy to test. It is the opposite of an agent, where the model chooses its own next action.
Prompt chaining is a good fit when the task splits into stages you can name before you see the input. Extracting numbers from a report, translating a document and then summarizing it, or drafting content and then checking it against a rule are all examples. If the subtasks depend on the input in ways you cannot predict, a planning pattern such as orchestrator-workers is a better choice. Chaining is the simpler and cheaper option when the stages are known.
The sample task
The sample takes a short quarterly report, written in plain prose, and turns the numbers in it into a sorted markdown table. The input looks like this:
Q3 Performance Summary:
Our customer satisfaction score rose to 92 points this quarter.
Revenue grew by 45% compared to last year.
Market share is now at 23% in our primary market.
Customer churn decreased to 5% from 8%.
New user acquisition cost is $43 per user.
Product adoption rate increased to 78%.
Employee satisfaction is at 87 points.
Operating margin improved to 34%.
The task is hard for a single prompt for a specific reason. The model must find the metrics, normalize mixed units ("92 points" and "45%" describe different things), sort them numerically, and then produce a table with exact formatting. Each of those is a small job. Together, they are easy to get subtly wrong.
Four steps, four prompts
The sample splits the work into four system prompts, stored as an array of text blocks in ChainWorkflow:
private static final String[] DEFAULT_SYSTEM_PROMPTS = {
// Step 1
"""
Extract only the numerical values and their associated metrics from the text.
Format each as 'value: metric' on a new line.
...""",
// Step 2
"""
Convert all numerical values to percentages where possible.
If not a percentage or points, convert to decimal (e.g., 92 points -> 92%).
Keep one number per line.
...""",
// Step 3
"""
Sort all lines in descending order by numerical value.
Keep the format 'value: metric' on each line.
...""",
// Step 4
"""
Format the sorted data as a markdown table with columns:
| Metric | Value |
|:--|--:|
..."""
};
I have shortened the examples in the snippet, but the structure is the same. Each step has one job. Step 1 extracts. Step 2 normalizes. Step 3 sorts. Step 4 formats. The prompts are short, and each one describes a single transformation with a concrete example of the expected output. That example is doing real work: it pins down the format that the next step will depend on.
The separation has a practical benefit. If the final table is wrong, you can look at the output of each step in the logs and see where the error entered. In the sample, the first step's output is a list of value: metric lines, and that list is the contract the second step depends on. If step 1 produces prose instead of lines, step 2 has nothing reliable to work with.
The chain loop
The execution logic is a plain loop. It is the part of the sample I like most, because it is short enough to read in one sitting:
public String chain(String userInput) {
int step = 0;
String response = userInput;
System.out.println(String.format("\nSTEP %s:\n %s", step++, response));
for (String prompt : systemPrompts) {
// 1. Compose the input using the response from the previous step.
String input = String.format("{%s}\n {%s}", prompt, response);
// 2. Call the chat client with the new input and get the new response.
response = chatClient.prompt(input).call().content();
System.out.println(String.format("\nSTEP %s:\n %s", step++, response));
}
return response;
}
Each iteration builds the input from the step's prompt and the previous response, calls the model, and replaces response with the result. The loop does not know what the steps mean. It only moves text from one step to the next. That is the core of the pattern: the chain is data flow, and the intelligence lives in the prompts.
Two details are worth noticing. First, the constructor accepts a custom array of prompts, so you can reuse the same loop for a different sequence of transformations. Second, the method returns only the final response. The intermediate outputs are printed but not returned, which is fine for a demo and a problem for a service that needs to audit what happened at each step.
The gates that the README describes but the code does not have
The README says each step acts as a gate that validates and transforms the output before the next step runs. The code does not do that. The loop passes every response forward without checking it. If step 1 returns an empty string or a paragraph of apology, step 2 receives it and tries its best.
I think this is the most important gap to close before you rely on the pattern. A gate is a small check between two steps. It can be a deterministic check, such as verifying that every line matches the pattern value: metric, or a model-based check, such as asking a second prompt whether the output is valid. If the gate fails, you have three reasonable options: retry the step with the same input, retry with the error message added to the prompt, or stop and return an error.
A minimal deterministic gate for step 1 could look like this:
private static final Pattern LINE = Pattern.compile("^[^:]+: .+$");
private static void requireLines(String output, int stepIndex) {
boolean valid = output.lines()
.filter(l -> !l.isBlank())
.allMatch(l -> LINE.matcher(l).matches());
if (!valid) {
throw new IllegalStateException("Step " + stepIndex + " produced unexpected output");
}
}
You would call it inside the loop after each step. The point is not this specific regex. The point is that every step has an explicit contract, and the code checks it before the next step runs.
Failure handling
The sample has no failure handling at all. If the model call throws, the exception propagates and the chain stops, which is the right default but not enough for production. In a real service I would add three things.
A retry policy for transient errors such as rate limits and timeouts, with a small number of attempts and exponential backoff. These errors are common with hosted models and rarely indicate a problem with the prompt.
A limit on total cost and time. A chain of four calls is four times the latency and four times the token spend of a single call, so a timeout for the whole chain is a good safety net.
Structured logging of the input and output of each step, at a level that does not leak user data. The loop already prints every step, which is useful for learning and too noisy for production. Replace the System.out.println calls with a logger and log the step index, the duration, and the output size. Log the full text only in a debug mode you control.
Testing the chain
The chain is easy to test because the loop does not depend on the model. You can pass a stub ChatClient that returns canned strings for each step, then assert on the final output and on the order of the calls. That tests the wiring without spending tokens.
Testing the prompts is a different problem. The prompts are the logic, and they depend on the model. The practical approach is a small set of representative inputs with known expected outputs, stored in version control, and rerun whenever you change a prompt or upgrade the model. Compare each step's output against its expected form, not just the final table, because the intermediate outputs are where drift appears first.
Designing the contract between steps
The most useful habit I have picked up from chains is to define the output of each step before I write its prompt. That contract is the interface between two prompts, and it deserves the same care as a method signature. If step 1 promises one value: metric pair per line, step 2 can be written with confidence. If step 1 is allowed to return anything, every later step has to defend against anything.
A good contract is specific enough to check. "Return the metrics" is weak. "Return one line per metric in the form value: metric, with no other text" is strong, because you can verify it with a regular expression, a parser, or a second model call. The examples inside each prompt do the same job for the model: they show the exact shape the next step expects.
Contracts also make refactoring safe. If you later replace step 2 with a deterministic function that converts units in Java, the rest of the chain does not need to change, as long as the function honors the same format. That is a practical reason to keep the steps small and their formats boring.
Choosing where the steps split
Splitting a task into steps is a judgment call, and I use two questions to make it. Does this stage need a different instruction than the previous one? Could a failure here be detected independently? If the answer to both is yes, the stage deserves its own step. If a stage only reformats what the previous one produced, it may be better as code than as a model call.
Deterministic steps are underused. Sorting numbers, removing duplicates, and building a markdown table from structured lines are all tasks that code does more reliably than a model. The sample uses the model for every step, which keeps the example uniform, but in a real system I would consider replacing steps 3 and 4 with plain Java. That would cut the cost, remove two sources of variance, and leave the model with the work that actually needs language understanding.
Chaining versus one prompt
Chaining is not free. Four calls cost more and take longer than one. The benefit is control: each step is simpler, more predictable, and easier to fix. I would start with a single prompt, measure how often it fails, and split it into a chain only when the failures have a clear shape that a step boundary could catch.
A useful rule is that a chain step should have one job and one output format. If a step needs to both extract and sort, split it. If two steps always run together and never need separate validation, merge them. The right number of steps is the smallest number that makes each failure easy to locate.
Running the sample
To run the sample you need Java 17 or later and a configured model. The project includes properties for OpenAI, Anthropic, and Ollama, so choose the one you use and set its key or start its local server. The README describes an Ollama setup, which runs the model locally and avoids API costs during experiments. Start the application with the Maven wrapper, and the console will print each step's output followed by the final table.
Try changing the input to a different report, or add a fifth step that rounds the percentages to whole numbers. Watching the intermediate outputs is the fastest way to build intuition for what each step contributes.
Key takeaways
- Prompt chaining splits one hard task into a fixed sequence of simple steps, each with one job and one output format.
- In Spring AI, the chain is a loop that feeds each response into the next prompt. The intelligence lives in the prompts, and the loop only moves data.
- Intermediate outputs are where failures are easiest to see. Log them, and test them.
- The sample does not implement the gates its README describes. Add a check between steps before you rely on the output.
- Add retries, a time limit and structured logging before the chain runs in a service.
The full sample is in the spring-ai-examples repository under agentic-patterns/chain-workflow. The pattern itself is described in Anthropic's research on building effective agents, which covers prompt chaining alongside the other workflows in this series.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes โ full credit and traffic to the original publisher.