Models / Mistral Small 22B

Best home rig for Mistral Small 22B

Legacy model1 measured resultfastest plain decode: 26.72 tok/s

Newer model available: Mistral Small 4. Mistral Small 22B is a legacy model; its numbers stay here for reference. Mistral Small 4 is 119B total, 6B active. source ↗

Real, measured generation speeds for Mistral Small 22B on home hardware, fastest first. Different quants/engines aren't directly comparable.

Build	Quant	Engine	tok/s	Cost	Source
RTX 4090 desktop (speed comparison vs Macs)	Q6_K	llama.cpp-based	26.72		someoddcodeguy.dev ↗

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

FAQ

What is the fastest measured hardware for Mistral Small 22B?

RTX 4090: 26.72 tok/s generation (llama.cpp-based, Q6_K), from https://www.someoddcodeguy.dev/low-context-speed-comparison-macbook-mac-studios-and-rtx-4090/.