Best home rig for Qwen3.5 9B
Will it fit my GPU? Best value GPU per euro →
Real, measured generation speeds for Qwen3.5 9B on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Cost | |
|---|---|---|---|---|---|
| RTX 5090: 2026 models (llamabench community) | llama-bench tg128 (quant per result) | llama.cpp | 208 | source | |
| RTX 4090: 2026 models (llamabench community) | llama-bench tg128 (quant per result) | llama.cpp | 137 | source | |
| Mac Studio M5 Max 36 GB: Qwen3.6 35B-A3B at 173 tok/s (Rapid-MLX), 146 on oMLX | 4-bit MLX | Rapid-MLX 0.16.1 (MTP) | 115.5 | source | |
| Mac Studio M5 Max 36 GB: Qwen3.6 35B-A3B at 173 tok/s (Rapid-MLX), 146 on oMLX | 4-bit MLX | oMLX 0.7.0 | 78.0 | source |
Memory math: how much VRAM / RAM does Qwen3.5 9B need?
Qwen3.5 9B has 8.95B parameters (source: GGUF param count via llamabench.ai). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.
| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 9.5 GB | 12 GB |
| Q6_K | 6.56 | 7.3 GB | 8 GB |
| Q5_K_M | 5.70 | 6.4 GB | 8 GB |
| Q4_K_M | 4.89 | 5.5 GB | 8 GB |
| Q3_K_M | 4.00 | 4.5 GB | 8 GB |
| Q2_K | 3.16 | 3.5 GB | 8 GB |
Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.
FAQ
How much VRAM does Qwen3.5 9B need?
The weights are about 5.5 GB at Q4_K_M (8.95B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX 4090 (24 GB) at 137 tok/s.
What is the fastest measured hardware for Qwen3.5 9B?
RTX 5090: 208 tok/s generation (llama.cpp, llama-bench tg128 (quant per result)), from https://llamabench.ai/browse/rtx5090.
Which GPUs have measured results for Qwen3.5 9B?
RTX 4090, RTX 5090.
Can it run on… RTX 4090 · RTX 5090
Related: what GPU do you need for 2026 models · all builds