Best home rig for Llama 3.1 70B

Older model. For current picks see 2025–2026 models.

Real, measured generation speeds for Llama 3.1 70B on home hardware, fastest first. Different quants/engines aren't directly comparable.

Build	Quant	Engine	tok/s
Mac Studio M3 Ultra 512GB (Geerling test)		llama.cpp	14.08	source
Mac Studio M1 Ultra (64-core GPU) 128GB		llama.cpp	9.84	source
EPYC 7F52 jank rig: 4x3090 + 4090 + 4080 (136 GB VRAM)	Q8_0	Oobabooga	9	source ⚠ unchecked
Framework Desktop (AMD Strix Halo) mainboard	Q4_K_M	llama.cpp Vulkan (iGPU, llama-bench tg128)	4.97	source
Dell Pro Max with GB10 (Nvidia DGX Spark)		llama.cpp	4.71	source
AmpereOne A192-32X 512GB (CPU only)		llama.cpp	3.86	source
Ryzen 9 7900X + RTX 4090		GPU+CPU offload	3.1	source

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Compare: DGX Spark vs Mac Studio vs Strix Halo: measured tok/s and tok/s per watt

FAQ

Apple M3 Ultra: 14.08 tok/s generation (llama.cpp), from https://github.com/geerlingguy/ai-benchmarks/issues/42.