How Attention Works: I Read All 24 Heads of a Tiny Language Model and Found the One That Watches the Main Character's Name
Part 2 of ten. Each post is a short, plain-words tour of one lesson from e=mc², a free build-an-LLM-from-scratch course; the full lesson, notebook and numbers are on the site. Attention is the part of a language model t
Part 2 of ten. Each post is a short, plain-words tour of one lesson from e=mc², a free build-an-LLM-from-scratch course; the full lesson, notebook and numbers are on the site.
Attention is the part of a language model that lets each word look back at the words before it, and yesterday's tiny model is small enough that you can watch it happen. In Lesson 01 I trained a 5.9-million-parameter storyteller on a free Colab GPU. Today nothing gets trained. I open those same weights, switch on one flag that records who looked at whom, and read all 24 of its attention heads. One of them, it turns out, spends about 22% of its attention on the main character's name.
Momo and Ella, again
Momo the monkey is the model; his dials were set last lesson and stay put tonight. Ella the elephant is the teacher and the memory. She reads him the story, and she keeps a shelf of everything already said. Picture the story told in a circle: each word has a seat, and each word may look at the words that spoke before it, never at the ones still to come.
Attention is the looking.
A question, a card and a handover
Here is the whole idea. Inside one attention head, every word gets three short lists of numbers, 64 numbers each in this model.
The first list is a query: the question a word asks. When "He" comes up in a story, the question is roughly "who is he?"
The second is a key: the card a word holds up for every later word to see. "Momo" holds up a card that says, in effect, "I am a name."
The third is a value: what the word hands over if it gets picked.
Now suppose I am the word "He". I compare my question to every earlier word's card. The better the match, the higher the score. A step called softmax turns those scores into shares that add up to 100%, and I take the share-weighted blend of everyone's values. If most of my shares went to "Momo", most of what I carry forward came from "Momo".
Write down those shares for every word in a story and you get an attention map: a table of who looked at whom.
One honest note: nothing is asked in words. A query is 64 numbers made by a matrix multiplication, and "looking for a name" is a label I read into a head afterwards. Momo was only ever trained to guess the next word.
The triangle, and Ella's shelf
Momo must never peek at words after the one he is guessing. The causal mask enforces it: before the softmax, every score for a later word is set to minus infinity, so its share is exactly zero. Lay a whole story's map out and it is a lower triangle. The dark upper-right corner of every heatmap is the rule, not a decision the head made.
The mask also explains the KV cache, which is just Ella's shelf. Earlier words cannot see later ones, so an earlier word's key and value never change once computed. When Momo writes a story one token at a time, Ella files each token's key and value on the shelf, and the next token computes only its own and reads the rest. Writing 200 tokens costs 200 key-and-value computations instead of 20,100. In the notebook, the shelf's answer matches recomputing everything within 0.0000003. It is a speed trick, not a memory of the story: what Momo knows lives in his dials, not on the shelf.
Reading a real model's mind
The notebook copies the model from Lesson 01 and adds one flag. With the flag on, each block computes attention the long way and keeps the shares; with it off, it uses PyTorch's fast built-in routine as before. The two agree within 0.00002 in the raw output scores, and every row of every map sums to 1. That flag is the whole change:
def forward(self, x):
B, T, C = x.shape
q, k, v = self.qkv(x).split(C, dim=2) # a query, key and value per token
q, k, v = [t.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) for t in (q, k, v)]
if self.return_attn: # the only change from Lesson 01
scores = (q @ k.transpose(-2, -1)) / math.sqrt(k.size(-1))
mask = torch.tril(torch.ones(T, T, dtype=torch.bool, device=x.device))
att = F.softmax(scores.masked_fill(~mask, float("-inf")), dim=-1)
self.last_attn = att # (batch, 4 heads, T, T): who looked at whom
y = att @ v
else:
y = F.scaled_dot_product_attention(q, k, v, is_causal=True)...
Then I plot the triangle, plot all 24 heads on one story, and measure every head over the 200 quiz stories Momo never trained on. A main character, by a deliberately crude rule, is the most frequent capitalised name in a story, mentioned at least twice; 159 of the 200 stories have one. For each head I add up how much attention the other words, after the first mention, send back to that name.
The numbers, honestly
- 24 heads read: 6 blocks of 4 heads, all from the Lesson 01 weights. No training.
- Even-spread baseline: 0.052. If a head spread its attention evenly, that is roughly how much would land on the name.
- The winner, block 5, head 3 (L5H3): 0.221. About 22% of its attention on the main character's name, 4.3 times an even spread. L4H4 (0.199) and L6H2 (0.178) are close behind.
- 17 of the 24 heads put less on the name than an even spread would.
- Previous-token heads: 0 of 24. None puts more than half its attention there; the highest is L1H1 at 0.146. That was the disappointing result, and I am reporting it as is.
- 8 to 20 seconds for the whole notebook on an Apple Silicon Mac; it also runs on a free Colab T4 or a plain CPU.
What it does not mean
Three limits, said plainly. L5H3 puts about 22% of its attention on the name, not all of it; this is a tendency, not a switch. Attention is where a head looks, not proof of what the model relies on; I did not switch any head off to see what breaks. And name position is not controlled: a name early in a story may draw attention partly for being early.
So no, this does not prove Momo understands who the hero is. It shows, with a real number from a real model, where one of his 24 ways of looking tends to go. That is a smaller claim and a truer one, and it is the first step toward explaining the 61-word story Momo recited in Lesson 01 and could not explain.
Why this is the lesson that matters
Every large model you have heard of is this mechanism with more heads, more blocks and more zeros. If you have ever wondered how a chat model keeps track of a character across a paragraph, you now have the picture: a circle of words looking back, and an elephant with a shelf.
The full lesson has the glossary table, the heatmaps of all 24 heads, a clickable toy widget, the ranked table of every head and the notebook that reproduces the 0.221.
Want to actually build it?
This post is the postcard. The lesson is the trip. Lesson 02 of e=mc² on theaivibe.org walks you through every piece with widgets you can poke, a notebook that runs on a free GPU in minutes, and the honest numbers at every step. No account, no paywall, nothing to install.
- Start here: https://theaivibe.org/emc2/how-attention-works-tiny-storyteller-attention-maps
- The whole ten-lesson journey: https://theaivibe.org/emc2
Tomorrow: Momo recited a 61-word story word for word in Lesson 01, so I train the same recipe at four sizes and watch for the moment memorising begins.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.