EPYC 7532 workstation (RTX Pro 6000 + RTX 3090 Ti, 300 W caps): Qwen3.8-Flash-Next, llama.cpp vs Strata engine
| GPU(s) | RTX Pro 6000 Blackwell Workstation (96 GB) + RTX 3090 Ti (EVGA FTW3 Ultra, 24 GB); each test used one card Amazon · eBay (used) · Newegg |
|---|---|
| VRAM / unified memory (GB) | 120 |
| CPU | AMD EPYC 7532 (32 cores, Zen 2) Amazon · eBay (used) · Newegg |
| Motherboard | |
| RAM | 128 GB DDR4-3200, 8 channels Amazon · eBay (used) · Newegg |
| PSU | |
| Other | Both cards power-capped to 300 W for the bench; one request at a time, 4,038-token prompt, 256 tokens out, greedy |
| Reported total cost | |
| Power | Board power (not wall), 4K prompt: RTX Pro 6000 297.8 W (llama.cpp) / 290.6 W (Strata); RTX 3090 Ti 168.5 W (llama.cpp) / 281.1 W (Strata) |
Measured speed (generation tokens/sec)
📏 How measured: reported by the original source; RigBase re-checked the numbers at that source
| Model | Quant | Engine | tok/s | Details |
|---|---|---|---|---|
| Qwen3.8 Flash-Next | IQ3_S, all experts on the RTX Pro 6000, no draft head | llama.cpp (3cf03257), RTX Pro 6000 | 102.889 | ⚡ spec-decode (speculative) |
| Qwen3.8 Flash-Next | IQ3_S + model draft head, all experts on the RTX Pro 6000 | Strata engine 0.1.39, RTX Pro 6000 | 205.263 | ⚡ spec-decode (speculative) |
| Qwen3.8 Flash-Next | IQ3_S, 32K prompt, all on the RTX Pro 6000 | llama.cpp (3cf03257), RTX Pro 6000 | 76.337 | |
| Qwen3.8 Flash-Next | IQ3_S, experts of 34/48 layers in system RAM, no draft head | llama.cpp (b11514), RTX 3090 Ti | 22.657 | ⚡ spec-decode (speculative) |
| Qwen3.8 Flash-Next | IQ3_S + draft head, 8,144 of 24,576 experts on card | Strata engine 0.1.39, RTX 3090 Ti | 98.291 | ⚡ spec-decode (speculative) |
strata→signal research, published 12 Oct 2026; runs 6-9 Oct 2026. Median of three runs. llama.cpp ran near defaults (512-token micro-batch, 8 threads); on the 3090 Ti its speed was limited by those 8 CPU threads, not the card. Strata is a third-party engine built on llama.cpp (github.com/Niko1221/Strata). The source also benches an RTX 5090 laptop GPU (24.4 tok/s llama.cpp, 85.6 Strata).
Source: https://research.strata2signal.com/flash-next-on-three-cards/
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.
Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)