Best home rig for Gemma 4 26B-A4B
Will it fit my GPU? Best value GPU per euro →
Real, measured generation speeds for Gemma 4 26B-A4B on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Cost | |
|---|---|---|---|---|---|
| RTX 5090: 2026 models (llamabench community) | llama-bench tg128 (quant per result) | llama.cpp | 235 | source | |
| MacBook Pro M5 Max 128 GB: Qwen3.6 35B-A3B at 271 tok/s (mlx-serve + MTP), Gemma 4 26B-A4B at 166 | 4-bit MLX (QAT) | Rapid-MLX 0.16.1 + DFlash | 166 | source | |
| RTX 4090: 2026 models (llamabench community) | llama-bench tg128 (quant per result) | llama.cpp | 164 | source | |
| Single Intel Arc Pro B70 32 GB (neural.download lab) | UD-Q8_K_XL | llama.cpp SYCL (pinned build, oneAPI 2026.0) | 122.16 | source | |
| DGX Spark (GB10, 128 GB): 23 models on vLLM at 128K context, Qwen3.6 35B-A3B up to 102 tok/s | + official MTP drafter (k=3) | vLLM 0.31.0, 128K ctx, 1 user | 56.5 | source | |
| HP ZBook Ultra G1a (Strix Halo laptop, 128 GB): Qwen3.6 35B-A3B at 59.4 tok/s, gpt-oss 120B at 44.7 | 4-bit | Ollama 0.34.0 | 51.8 | source | |
| DGX Spark (GB10, 128 GB): 23 models on vLLM at 128K context, Qwen3.6 35B-A3B up to 102 tok/s | no speculative decoding | vLLM 0.31.0, 128K ctx, 1 user | 30.8 | source |
Memory math: how much VRAM / RAM does Gemma 4 26B-A4B need?
Gemma 4 26B-A4B has 25.23B parameters, about 4B active per token (source: GGUF param count via llamabench.ai). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.
| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 26.8 GB | 32 GB |
| Q6_K | 6.56 | 20.7 GB | 24 GB |
| Q5_K_M | 5.70 | 18.0 GB | 24 GB |
| Q4_K_M | 4.89 | 15.4 GB | 16 GB |
| Q3_K_M | 4.00 | 12.6 GB | 16 GB |
| Q2_K | 3.16 | 10.0 GB | 12 GB |
Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.
Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 4B of its 25.23B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.
FAQ
How much VRAM does Gemma 4 26B-A4B need?
The weights are about 15.4 GB at Q4_K_M (25.23B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX 4090 (24 GB) at 164 tok/s.
What is the fastest measured hardware for Gemma 4 26B-A4B?
RTX 5090: 235 tok/s generation (llama.cpp, llama-bench tg128 (quant per result)), from https://llamabench.ai/browse/rtx5090.
Which GPUs have measured results for Gemma 4 26B-A4B?
DGX Spark (GB10), Intel Arc Pro B70, RTX 4090, RTX 5090, Strix Halo (Radeon 8060S).
Can it run on… DGX Spark (GB10) · Intel Arc Pro B70 · RTX 4090 · RTX 5090 · Strix Halo (Radeon 8060S)
Related: what GPU do you need for 2026 models · all builds