Picking the right illustration without looking at a single image

Our library has a picture on every page — tens of thousands of them. When a story gets shared, one of those pictures has to stand for the whole book: the og:image, the link preview, the card on someone else's feed.

For a long time we got that wrong in a way nobody noticed, because the bug only appears on a subset of books.

The bug

Our page generator did this:

const cover = coverOf(book);
// …
decorateShell({ title, description, url, image: cover, … });

coverOf() walks a priority list — coverVariants, then legacy variants, then coverUrl, coverImage. For a book with a cover, that's correct and has always been correct.

For a book without one, it fell through to the first page illustration.

Page 1 of almost any story is the worst possible choice. Page 1 is the ordinary world. It's the kitchen before anything happens, the workshop before the invention, the village before the traveller arrives. In Cinderella, page 1 is a girl in ashes carrying water. That was the image representing the story everywhere it was shared.

The founder spotted it before any of our checks did:

"If you will pick a picture, pick the big picture, the best illustration about the story that represents the story. Not this one. This is when she was poor. Why don't you show when she was in the palace dancing?"

The obvious fix is the expensive one

The reflex is to call a vision model: send it ten images, ask which best represents the story. That's one paid call per book, across a catalogue of thousands, re-run whenever the pipeline changes. It also returns a different answer on different days.

We had a standing rule against exactly this — don't analyse the images.

So the constraint was: choose the right picture without looking at any picture.

What we did instead

The illustrations are generated from the page text. The prose is therefore a free and surprisingly honest proxy for what its picture shows. If page six says "in a swirl of golden sparks it swelled and gilded and rounded into a splendid golden coach", you can be confident page six's illustration is a golden coach.

So we score the text:

const SPECTACLE = [
  'golden', 'blaze', 'glittering', 'chandelier', 'lanterns',
  'palace', 'ballroom', 'throne', 'temple', 'vast', 'towering',
  'crowd', 'procession', 'feast', 'coronation',
  'gown', 'crown', 'armour', 'slippers', 'transformed',
  'dancing', 'soaring', 'storm', 'lightning', 'battle',
  'discovered', 'invention', 'telescope', 'launched', 'unveiled',
];

const QUIET = [
  'once', 'there lived', 'at the edge', 'every morning',
  'swept', 'scrubbed', 'mended', 'chores', 'waited', 'alone',
];

These aren't "good writing" words. They're words that mean the illustration for that page is a spectacle: a crowd, a hall, light, scale, costume, motion. A page of quiet dialogue gets a quiet picture however fine the prose is.

Then a position term, because a story's peak is not uniformly distributed:

const where = total > 1 ? (n - 1) / (total - 1) : 0;
const position = Math.exp(-((where - 0.6) ** 2) / (2 * 0.22 ** 2));

A bell centred at 60% through the book — past the setup, at or just before the climax, short of the ending.

And two hard floors, which came straight from the founder's own words — "let's not put the last one, at least the one in the middle":

const banned = n === 1 || n === total;
return { n, score: banned ? -1 : spectacle - quiet * 0.6 + position * 6 };

Never page one. Never the ending — nobody wants the last page of a story in its own preview.

Does it work?

On Cinderella it returns page 6 — the pumpkin becoming a coach, the gown, the glass slippers. That is the exact page he named. Page 7, the ballroom, scores second; it's the other page he named. Pages 1 and 10 are excluded by rule.

page  6  score 91.97  spectacle 86.1  quiet 0  pos 0.98
page  7  score 71.47  spectacle 69.9  quiet 7  pos 0.96
page  8  score 61.47  spectacle 64.9  quiet 13 pos 0.72
page  3  score 25.32  spectacle 28.2  quiet 7  pos 0.23

Across a sample of 48 books from twelve collections, 44 had a real cover and were left completely untouched — the rule is that a cover always wins, because somebody chose it and no heuristic beats a decision. Only the coverless ones changed, and they changed from "the ordinary world" to the story's peak.

Three things worth taking from this

A fallback is a decision nobody made. coverOf() returning page 1 was never designed. It was what happened when a priority list ran out of options. Fallbacks are where defects hide, because they only fire on the subset of your data that lacks something.

The cheap signal is often sitting right there. The images were generated from the text. Reaching for a vision model would have meant paying to re-derive information we already had in a column.

"No image" and "an image I shouldn't use" are different states. Our code treated them identically. The fix includes isRealCover(url) — a cover lives under /covers/, and anything else that arrives in the cover field is a page illustration something upstream mislabelled. Without that check, a coverless story quietly inherits page 1 all over again.

WonderBooks is a library of illustrated, narrated stories — twelve collections, readable in thirteen languages, free on the web, iOS and Android: https://thewonderbooks.com/