Few-shot examples teach format, not facts: how to pick, balance and order them

Monday AI Insight. Most of us add few-shot examples to a prompt the same way: grab three or four that look right, paste them in, ship. It works often enough that we never ask what the model is actually learning from them.

The research answer is a little uncomfortable: the model mostly learns the shape of the task from your examples, not the right answers. And it picks up your mistakes in label balance and ordering along the way.

Here's what three well-known papers found, and the practical rules I take from them.

1. Examples teach the format and the label space

Min et al. (2022), Rethinking the Role of Demonstrations, tested 12 models (including GPT-3) on classification and multiple-choice tasks. They swapped the correct labels in the demonstrations for random labels, and performance barely moved: drops were mostly in the 0–5% absolute range.

What did matter:

• The label space: showing the model which answers are allowed.
• The input distribution: examples that look like the real inputs. Swapping in out-of-distribution text dropped performance by 3–16% absolute for several models.
• The format: input followed by label, the same way every time. Removing the input-label pairing was close to, or worse than, no examples at all.

They also found gains flattened once there were about 8 or more examples.

Takeaway: your examples are mostly a format spec and a menu of valid outputs. Make them look exactly like production inputs, and make the output format identical across all of them.

(One honest caveat: the same paper notes the gap was larger on some datasets, and later work found that deliberately wrong labels do hurt. Correct labels still matter; they just aren't the main thing.)

2. The model copies whatever is most common and most recent

Zhao et al. (2021), Calibrate Before Use, found three biases in few-shot prompts:

• Majority label bias: if most examples share a label, the model over-predicts it.
• Recency bias: the model leans toward the label of the examples near the end. In one 4-shot sentiment setup, a "Positive, Positive, Positive, Negative" order led to nearly 90% Negative predictions, even though three of four examples were Positive.
• Common token bias: the model prefers answers that are frequent in its pre-training data.

The same paper showed that changing only the order of examples moved accuracy from 54.3% to 93.4% on a sentiment task, and that adding more examples didn't reliably remove the variance.

Takeaway: balance your labels, and don't put all examples of one class at the end.

3. Order matters, and good orders don't transfer between models

Lu et al. (2022), Fantastically Ordered Prompts, ran all 24 orderings of the same four examples. Some orderings got near state-of-the-art accuracy; others were close to random guessing. Worse, a good order for one model didn't predict a good order for another: one permutation fell from 88.7% to 51.6% just by changing the model size.

Takeaway: the ordering you hand-tuned last quarter is not guaranteed to survive a model upgrade. Re-check it with evals when you switch models.

A practical checklist

1. Copy real inputs. Use examples drawn from (or very close to) real production inputs, including messy ones.
2. Keep the format identical. Same delimiters, same field names, same output shape in every example. Wrapping each one in clear tags (Anthropic's docs suggest <example> tags) helps the model separate examples from instructions.
3. Cover the label space. Show every allowed output at least once, including the boring ones like "none" or "not applicable".
4. Balance the labels. Avoid 4 positives and 1 negative unless that's truly what you want the model to lean toward.
5. Mix the order. Don't cluster one class at the end. Interleave.
6. Stay small. A handful of diverse examples beats a long list of similar ones. Anthropic's prompting guide recommends 3–5 diverse, relevant examples as a starting point.
7. Measure, don't eyeball. Run your eval set with two or three different orderings. If accuracy swings a lot, your prompt is fragile, and you just learned that before your users did.

Example: a support ticket classifier

Bad:

Ticket: "Card charged twice" -> billing
Ticket: "Refund not received" -> billing
Ticket: "Invoice has wrong GST" -> billing
Ticket: "App crashes on login" -> bug

Three billing examples, one bug, and no "other" label. The model learns that "billing" is the safe default, and it has never seen "feature_request" or "other".

Better:

<example>Ticket: "App crashes on login" -> bug</example>
<example>Ticket: "Card charged twice" -> billing</example>
<example>Ticket: "Please add dark mode" -> feature_request</example>
<example>Ticket: "What are your office hours?" -> other</example>

One example per label, realistic wording, identical format, mixed order.

The one-line version

Few-shot examples are a format spec, not a training set. Pick examples that look like real inputs, cover every label, balance them, mix the order, and test more than one ordering before you trust the prompt.

What's the strangest thing a few-shot example ever taught your model by accident?

References

• Min et al., 2022. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? https://arxiv.org/abs/2202.12837
• Zhao et al., 2021. Calibrate Before Use: Improving Few-Shot Performance of Language Models. https://arxiv.org/abs/2102.09690
• Lu et al., 2022. Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity. https://arxiv.org/abs/2104.08786
• Anthropic docs: Use examples (multishot prompting). https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/multishot-prompting