Best home rig for LFM2.5 24B-A2B

Will it fit my GPU? Best value GPU per euro →

Real, measured generation speeds for LFM2.5 24B-A2B on home hardware, fastest first. Different quants/engines aren't directly comparable.

Build	Quant	Engine	tok/s
RTX 4090: 2026 models (llamabench community)	llama-bench tg128 (quant per result)	llama.cpp	261-339	source
RTX 4060 Ti 16GB: 2026 models (llamabench community)	llama-bench tg128 (quant per result)	llama.cpp	131-150	source
RTX 3060 12GB: 2026 models (llamabench community)	llama-bench tg128 (quant per result)	llama.cpp	125	source

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Memory math: how much VRAM / RAM does LFM2.5 24B-A2B need?

LFM2.5 24B-A2B has 23.97B parameters, about 2.45B active per token (source: via llamabench.ai). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

Format	Bits/weight	Weights	Smallest common memory size that holds the weights alone
Q8_0	8.50	25.5 GB	32 GB
Q6_K	6.56	19.7 GB	24 GB
Q5_K_M	5.70	17.1 GB	24 GB
Q4_K_M	4.89	14.7 GB	16 GB
Q3_K_M	4.00	12.0 GB	12 GB
Q2_K	3.16	9.5 GB	12 GB

Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.

Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 2.45B of its 23.97B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.

Measured on setups with less GPU memory than the Q4_K_M weights (so part of the model lived in system RAM or was streamed from disk):

RTX 3060 (12 GB): 125 tok/s (llama-bench tg128 (quant per result))

FAQ

How much VRAM does LFM2.5 24B-A2B need?

The weights are about 14.7 GB at Q4_K_M (23.97B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX 4060 Ti 16GB (16 GB) at 131-150 tok/s.

What is the fastest measured hardware for LFM2.5 24B-A2B?

RTX 4090: 261-339 tok/s generation (llama.cpp, llama-bench tg128 (quant per result)), from https://llamabench.ai/browse/rtx4090.

Which GPUs have measured results for LFM2.5 24B-A2B?

RTX 3060 12GB, RTX 4060 Ti 16GB, RTX 4090.

Can it run on… RTX 3060 12GB · RTX 4060 Ti 16GB · RTX 4090