Models / Llama 2 7B
Best home rig for Llama 2 7B
Legacy model2 measured resultsfastest plain decode: 188.96 tok/s
Newer model available: Muse Glimmer 30B. Llama 2 7B is a legacy model; its numbers stay here for reference. Meta’s current open models are Muse Glimmer 30B and Muse Spark Open 816B-A42B. source ↗
Real, measured generation speeds for Llama 2 7B on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Cost | Source |
|---|---|---|---|---|---|
| Single RTX 4090 (llama.cpp CUDA scoreboard) | Q4_0 | llama.cpp CUDA (FA) | 188.96 | github.com ↗ | |
| Mac Studio M2 Ultra 76-core GPU, 192 GB | Q4_0 | llama.cpp Metal (FA, Aug 2026 build) | 125.21 | github.com ↗ |
Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)
FAQ
What is the fastest measured hardware for Llama 2 7B?
RTX 4090: 188.96 tok/s generation (llama.cpp CUDA (FA), Q4_0), from https://github.com/ggml-org/llama.cpp/discussions/15013.
Related: what GPU do you need for 2026 models · all builds