Best home rig for Llama 3.3 70B
Older model. For current picks see 2025–2026 models.
Real, measured generation speeds for Llama 3.3 70B on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Cost | |
|---|---|---|---|---|---|
| Dual used RTX 3090, ~2000 EUR total | ExLlamaV2 (TP) | 20 | €2,000 (≈$2,242) | source ⚠ unchecked | |
| 2x RTX 3090 (llm-speed signed run, Oct 2026): Llama 3.3 70B Q4_K_M at 19.1 tok/s | Q4_K_M | llama.cpp 0.7.0-dev | 19.1 | source | |
| Mac Studio 64 GB | Q4 | 11 | $4,000 (≈€3,569) | source ⚠ unchecked | |
| RTX 3090 + RTX 4060 Ti 16GB mixed pair | IQ4_XS | 11 | source ⚠ unchecked | ||
| Single RTX 4060 Ti 16GB + 64 GB RAM (partial offload) | Q5_K_M | LM Studio / Kobold | 1.5 | source ⚠ unchecked |
Memory math: how much VRAM / RAM does Llama 3.3 70B need?
Llama 3.3 70B has 70B parameters (source: Ollama library tag 70b). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.
| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 74.4 GB | 96 GB |
| Q6_K | 6.56 | 57.4 GB | 64 GB |
| Q5_K_M | 5.70 | 49.9 GB | 64 GB |
| Q4_K_M | 4.89 | 42.8 GB | 48 GB |
| Q3_K_M | 4.00 | 35.0 GB | 48 GB |
| Q2_K | 3.16 | 27.6 GB | 32 GB |
Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.
Measured on setups with less GPU memory than the Q4_K_M weights (so part of the model lived in system RAM or was streamed from disk):
- RTX 3090 + RTX 4060 Ti 16GB (40 GB): 11 tok/s (IQ4_XS)
- RTX 4060 Ti 16GB (16 GB) + 64 GB: 1.5 tok/s (Q5_K_M)
FAQ
How much VRAM does Llama 3.3 70B need?
The weights are about 42.8 GB at Q4_K_M (70B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is 2x RTX 3090 (48 GB) at 20 tok/s.
What is the fastest measured hardware for Llama 3.3 70B?
2x RTX 3090: 20 tok/s generation (ExLlamaV2 (TP)), from https://www.reddit.com/r/LocalLLaMA/comments/1hdpvpt/llama_33_speed/.
Related: what GPU do you need for 2026 models · all builds