Best home rig for Llama 3.1 405B
Older model. For current picks see 2025–2026 models.
Real, measured generation speeds for Llama 3.1 405B on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Cost | |
|---|---|---|---|---|---|
| Mac Studio M3 Ultra 512GB (Geerling test) | llama.cpp | 1.96 | source | ||
| AmpereOne A192-32X 512GB (CPU only) | llama.cpp | 0.9 | source |
Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)
FAQ
What is the fastest measured hardware for Llama 3.1 405B?
Apple M3 Ultra: 1.96 tok/s generation (llama.cpp), from https://github.com/geerlingguy/ai-benchmarks/issues/42.
Related: what GPU do you need for 2026 models · all builds