GPT Explains GPT: A Visual Walk from Tokens to Multimodal Models

I used GPT to help write a Manim visual essay about language and multimodal models. The full film runs 8m57s, uses Chinese captions, original synthesized music and light sound effects, and has no narration.

Full video: https://www.bilibili.com/video/BV1apWd6uEJ9/

Start with a sentence

The animation begins with a next-token prediction problem. It turns text into tokens and numerical representations, adds position information, then shows attention retrieving and combining information from context. Only after that visible operation do the names Q, K and V appear.

This is a teaching model: token boundaries need not match words, learned attention is not a fixed keyword rule, and its weights are not a complete explanation of a model’s reasoning.

Build a language model

The film follows a common autoregressive decoder route: layers update representations; the model predicts a token, adds it to the context, and repeats. It then visualizes pretraining loss and parameter updates, followed by instruction-oriented post-training. Model size, data and compute all matter; scale alone is not a guarantee of correctness.

Make multimodality concrete

Over half the running time covers images, audio, video, alignment and outputs. Visual features plus a connector illustrate one representative vision-language route. Audio adds time-varying signals and potentially audio tokens and a decoder; video adds frame order and timing. Training connects those representations. Image understanding and image generation are treated as different capabilities.

The goal is to show the operation before introducing terminology, and to leave time to observe each transformation.

What the film illustrates

Every vector, probability, training curve, attention link, waveform and spectrogram is a teaching illustration, not a trace from GPT. The noise-to-cup shot is a fixed interpolation, not actual diffusion sampling. The film does not claim that one architecture describes every multimodal model.

AI-assisted writing and coding. Animation, music and sound effects were made in code.

Primary references

• Attention Is All You Need: https://arxiv.org/abs/1706.03762
• Training Compute-Optimal Large Language Models: https://arxiv.org/abs/2203.15556
• Visual Instruction Tuning: https://arxiv.org/abs/2304.08485
• Qwen2.5-Omni Technical Report: https://arxiv.org/abs/2503.20215