LLMs Locally on Consumer Hardware — Part 10: How the Setup Evolved — A Timeline of What Changed and Why

Part 10 of a series — a retrospective appendix. The narrative ran across Parts 1–9; this one lays the evolution out stage by stage.

No part of the setup was chosen for its own sake. Every change — a new machine, a different coordinator, a smaller model — was forced by a specific failure in the stage before it. Below is each stage as a snapshot, with the arrow between snapshots showing what broke and what was added or removed to fix it.

The timeline, stage by stage

Stage 0 — Initial

⬇ Broke: gemma4:26b can't hold a coordination loop beyond ~1–2 delegations.
Added: a search for a dedicated coordinator model (workers stay on gemma).

Stage 1 — Coordinator search (still one machine)

⬇ Broke: the 120B (~77 GB) and the gemma workers (~20 GB) can't both be resident in 80 GB RAM → ~100 GB swapped per hand-off; 291 s to say hello.
Removed: the workers from the main machine · Added: a second machine (5600G + GTX 1060).

Stage 2 — Distributed (two machines)

⬇ Noticed: with the 120B running on CPU and RAM, the main machine's GPU sits idle.
Added: image generation (ComfyUI + SDXL) on the main 6900XT.

Stage 3 — Image generation added

⬇ Broke: the 120B leaves only ~3 GB RAM free → SDXL generations time out.
Removed: gpt-oss:120b · Added: gpt-oss:20b (~14 GB) in its place.

Stage 4 — Right-sized coordinator

⬇ Added over the same period: a cloud strategist (Sage, on Gemini 3 Flash) for the one role needing deep reasoning; the laptop as an SD 1.5 image node; and deterministic orchestration (a script owns the delegation loop, the model only plans and evaluates).

Stage 5 — Final

The laptop's detour

The 8840U laptop ran its own small arc inside the larger one.

⬇ Why: no discrete GPU, so prefill is hopeless for large agent prompts (Part 2).
Repurposed for the one job the hardware could do:

Initial versus final

The pattern

Read top to bottom, the chain has a shape. Nearly every change was one of two moves: placement — which machine a thing runs on — or right-sizing — how large each model needs to be for its job. The workers moved for placement; the coordinator shrank for right-sizing; the strategist went to the cloud because that was the right placement for the one role that needed deep reasoning. Not once did the answer turn out to be "buy a bigger model and hope." The capability was almost always already there in the hardware on hand; the work was in arranging it so each piece had room to do the thing it was good at.