Best home rig for Flint 40B-A2.5B

Will it fit my GPU? Best value GPU per euro →

Real, measured generation speeds for Flint 40B-A2.5B on home hardware, fastest first. Different quants/engines aren't directly comparable.

Build	Quant	Engine	tok/s
RTX 4090 (llm-speed signed run, Oct 2026): Flint 40B-A2.5B at 169-179 tok/s on llama.cpp	NVFP4	llama.cpp 0.7.0-dev (build 11803)	179.4	source
RTX 4090 (llm-speed signed run, Oct 2026): Flint 40B-A2.5B at 169-179 tok/s on llama.cpp	UD-Q3_K_XL	llama.cpp 0.7.0-dev (build 11803)	169.2	source
RTX 3090 (llm-speed signed run, Oct 2026): Flint 40B-A2.5B Q3_K_XL at 128.9 tok/s	UD-Q3_K_XL	llama.cpp 0.7.0-dev (build 11803)	128.9	source
MacBook Pro M4 Pro 48 GB: Flint 40B-A2.5B 4-bit at 127.9 tok/s (Kapitsa), 126.6 (rec), 79.6 (llama.cpp)	int4 MLX (RecursiveSI/flint-int4)	Kapitsa (PR #286)	127.9	source
MacBook Pro M4 Pro 48 GB: Flint 40B-A2.5B 4-bit at 127.9 tok/s (Kapitsa), 126.6 (rec), 79.6 (llama.cpp)	int4 MLX (same files)	rec 0.1.0 (Recursive)	126.56	source
MacBook Pro M4 Pro 48 GB: Flint 40B-A2.5B 4-bit at 127.9 tok/s (Kapitsa), 126.6 (rec), 79.6 (llama.cpp)	Q4_1 GGUF (RecursiveSI/flint-int4-gguf)	llama.cpp b11804, all layers on GPU, -fa on	79.57	source

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Memory math: how much VRAM / RAM does Flint 40B-A2.5B need?

Flint 40B-A2.5B has 39.8B parameters, about 2.5B active per token (source: 39.8B total / ~2.5B active MoE coding model (Recursive, Apache 2.0), per Hugging Face model card). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

Format	Bits/weight	Weights	Smallest common memory size that holds the weights alone
Q8_0	8.50	42.3 GB	48 GB
Q6_K	6.56	32.7 GB	48 GB
Q5_K_M	5.70	28.4 GB	32 GB
Q4_K_M	4.89	24.3 GB	32 GB
Q3_K_M	4.00	19.9 GB	24 GB
Q2_K	3.16	15.7 GB	16 GB

Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.

Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 2.5B of its 39.8B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.

Measured on setups with less GPU memory than the Q4_K_M weights (so part of the model lived in system RAM or was streamed from disk):

RTX 4090 (24 GB) + 504 GB: 179.4 tok/s (NVFP4)
RTX 4090 (24 GB) + 504 GB: 169.2 tok/s (UD-Q3_K_XL)
RTX 3090 (24 GB) + 1008 GB: 128.9 tok/s (UD-Q3_K_XL)

FAQ

How much VRAM does Flint 40B-A2.5B need?

The weights are about 24.3 GB at Q4_K_M (39.8B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here is RTX 4090 (24 GB) at 179.4 tok/s (NVFP4). The measured runs on it used NVFP4 / UD-Q3_K_XL, not Q4_K_M. At Q4_K_M the weights are about 24.3 GB, more than 24 GB, so Q4_K_M doesn't fully fit without offloading part of the model to system RAM.

What is the fastest measured hardware for Flint 40B-A2.5B?

RTX 4090: 179.4 tok/s generation (llama.cpp 0.7.0-dev (build 11803), NVFP4), from https://llm-speed.com/r/r_sw92j6kg7js.

Which GPUs have measured results for Flint 40B-A2.5B?

Apple M4 Pro, RTX 3090, RTX 4090.

Can it run on… Apple M4 Pro · RTX 3090 · RTX 4090