Where does a 70B model actually fit? Dual RTX 3090 vs DGX Spark 64GB vs 128GB
70B dense models are the point where a single consumer GPU stops being enough. Here is what the memory math says for the three setups people usually compare. The baseline: Llama 3.3 70B at Q4_K_M About 40.7 G
70B dense models are the point where a single consumer GPU stops being enough. Here is what the memory math says for the three setups people usually compare.
The baseline: Llama 3.3 70B at Q4_K_M
About 40.7 GB of weights. Add a KV cache and an activation buffer:
| Context | Approx. memory |
|---|---|
| 4K | ~46.1 GB |
| 8K | ~47.5 GB |
| 32K | ~55.7 GB |
| 128K | ~88.7 GB |
Option 1: two RTX 3090s (48 GB total)
- 4K: 46.1 of 48 GB, 96% of the pair. It loads, but there is no margin, so plan for a headless box.
- llama.cpp's auto-fit keeps ~1 GiB free per card, so its effective budget is ~46 GB. Pass
-ngl alland an explicit-cso a model that doesn't fit fails loudly instead of silently offloading layers to CPU. - No NVLink needed: in the default layer split only boundary activations cross cards.
- Want comfort? A generic Q3_K_M of a 70B at 8K is ~38.4 GB.
Option 2: DGX Spark 64GB
- Llama 3.3 70B Q4_K_M at 32K: ~55.7 GB, inside the comfortable line. This is "the dense-70B machine".
- GPT-OSS 120B does not fit: MXFP4 at 8K is ~66.5 GB, Q4_K_M at 8K is ~62.9 GB (98% of the pool, with the OS sharing it).
Option 3: DGX Spark 128GB
- Everything above, plus the 100B-class MoE releases: GPT-OSS 120B, GLM-4.5-Air Q4_K_M (~68.7 GB at 8K), Llama 4 Scout Q4_K_M (~70.7 GB at 8K).
- At 4K context, 116 of the 129 models in the index fit comfortably (111 on the 64GB, 103 on a 32GB RTX 5090).
Speed is a different story
Both Sparks have 273 GB/s. Generation reads the whole dense model per token, so Llama 3.3 70B Q4_K_M tops out around 7 tok/s on either. An RTX 5090 moves 1,792 GB/s, about 6.6x more, but only on models it can hold. MoE models read only active experts, which is why they feel much faster on a Spark.
Rule of thumb
- Dense 70B, short context, budget: dual 24GB cards.
- Dense 70B with long context: Spark 64GB.
- 100B+ MoE: Spark 128GB.
Full walkthroughs with commands:
- Dual 3090 + llama.cpp: https://quantized.uk/cookbook/dual-gpu-70b-llamacpp/
- DGX Spark 64GB vs 128GB: https://quantized.uk/cookbook/dgx-spark-64gb-vs-128gb/
- Try your own model/context: https://quantized.uk/tools/vram-calc/
All numbers are calculator estimates; if your measured usage differs, I'd like to hear it.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes โ full credit and traffic to the original publisher.