Temperature and Sampling: the LLM Creativity Dial

Why does the same prompt give different answers? Temperature. One number turns an LLM from "safe and repetitive" to "creative and risky" by reshaping the next-word odds before it picks. Drag the dial and watch.

🌡️ Reshape + sample: https://dev48v.infy.uk/ai/days/day9-temperature.html

The model outputs a distribution

At each step it produces a probability for every possible next word — "weather is ___" → 46% sunny, 22% cloudy, 14% rainy, plus a long tail. Choosing one is a separate step called sampling.

Temperature reshapes the odds

p = Math.pow(p, 1 / temperature);  // then renormalise

• T → 0: sharpens to the top word (near-greedy, deterministic, repetitive).
• T ≈ 1: as-is.
• T > 1: flattens — rare words get a real shot (creative, error-prone).

Then it samples weighted by the reshaped probabilities, so two runs differ at higher T.

top-k and top-p trim the tail

Pure temperature can still pick something absurd from the tail. top-k keeps only the k likeliest words; top-p (nucleus) keeps the smallest set summing to p (e.g. 0.9). Both cut the weird tail while keeping variety.

Match it to the task

Facts, code, extraction → temperature ≈ 0 (reproducible). Brainstorming, copy, fiction → 0.7–1.0. Set temperature OR top-p, not both hard.

Drag the dial — low = same word every time, high = scattered.