An 8B LLM on a 4-core laptop CPU, no GPU: 6 tokens/sec

I ran an 8-billion-parameter Llama-3 model (Q4_K_M) on an HP laptop with an i7-1165G7, 12 GB of RAM and no discrete GPU.

At 6 tokens/sec, output runs at roughly reading speed.

Three results surprised me.

The integrated GPU was slower. Iris Xe read prompts nearly 4× faster than the CPU but wrote answers 2.5× slower. Writing is limited by memory bandwidth, and an integrated GPU shares the CPU's memory.

The 8-bit KV cache cost more than it saved. It looked free in short benchmarks. On a real 1,700-token summarization task, the 16-bit cache finished in 173 seconds instead of 234 and ran 7 °C cooler.

Capping CPU power cost nothing. Cutting the short-term limit from 51 W to 22 W eliminated the 98 °C startup spikes, and writing speed was unchanged.

If you're interested in more information, the full write-up has hardware details, every benchmark table, the exact llama.cpp commands, and a systemd service for the power cap:

👉 Running an 8B SLM on HP Laptop CPU-Only