RTX 3090 for local LLMs

28 measured results from 18 builds, each linked to its source.

Memory	24 GB per card
Memory bandwidth	936 GB/s (memory bandwidth as listed on llamabench)
Typical EU price	≈€1,248 used (eBay sold median, 237 sales, typical €1,136–€1,334; PCPrice.Watch, 2026-10-24)

Can it run? DeepSeek R1 671B on RTX 3090 DeepSeek V4 Flash on 8x RTX 3090 Flint 40B-A2.5B on RTX 3090 GLM 4.7 Flash on RTX 3090 Qwen3 235B-A22B on 4x RTX 3090 Qwen3.6 27B on RTX 3090 Qwen3.6 35B-A3B on 2x RTX 3090 Qwen3.6 35B-A3B on RTX 3090 Qwen3.8 27B on RTX 3090 gpt-oss 20B on RTX 3090

All builds with the RTX 3090

Build	GPU(s)	Memory GB (total)	Results
8x RTX 3090 serving DeepSeek V4 Flash (LimeChain vLLM kit)	8x RTX 3090 24 GB	192	1
EPYC 7F52 jank rig: 4x3090 + 4090 + 4080 (136 GB VRAM) ⚠ unchecked	4x RTX 3090, RTX 4090, RTX 4080	136	1
4x RTX 3090 in a 4U rack case (AM5) ⚠ unchecked	4x Dell OEM RTX 3090	96	0
4x RTX 3090 homelab on Threadripper 2920X ⚠ unchecked	4x RTX 3090 (~$625 each, used)	96	0
4x RTX 3090 + 32-core Threadripper, 192 GB DDR4 ⚠ unchecked	4x RTX 3090	96	1
4x RTX 3090 + EPYC 7443, 256 GB DDR4 8-channel ⚠ unchecked	4x RTX 3090	96	1
4x RTX 3090 lab: Qwen3.6 on vLLM vs llama.cpp	4x RTX 3090	96	3
Dual RTX 3090 on Ryzen 3700X (X470) ⚠ unchecked	2x Zotac RTX 3090	48	2
Dual used RTX 3090, ~2000 EUR total ⚠ unchecked	2x RTX 3090 (used)	48	1
2x RTX 3090 (llm-speed signed run, Oct 2026): Llama 3.3 70B Q4_K_M at 19.1 tok/s	2x RTX 3090	48	1
RTX 3090 + RTX 4060 Ti 16GB mixed pair ⚠ unchecked	RTX 3090 + RTX 4060 Ti 16GB	40	1
EPYC + single RTX 3090 (hybrid) ⚠ unchecked	RTX 3090	24	1
Single RTX 3090 24GB	RTX 3090	24	1
RTX 3090: 2026 models (llamabench community)	RTX 3090	24	3
Single RTX 3090: Qwen3.6 35B-A3B (Giles Thomas)	RTX 3090	24	1
Single RTX 3090: Qwen3.8 27B at 78-140 tok/s with the OpenJet megakernel vs llama.cpp	RTX 3090	24	6
RTX 3090 (llm-speed signed runs, Oct 2026): gpt-oss 20B 207 tok/s, Qwen3.6 27B 43 tok/s	RTX 3090	24	3
RTX 3090 (llm-speed signed run, Oct 2026): Flint 40B-A2.5B Q3_K_XL at 128.9 tok/s	RTX 3090	24	1

All measured results

Hide speculative-decoding results ⚡ (MTP, DFlash, DSpark and similar draft-and-verify methods give much higher tok/s on predictable text and are not comparable 1:1 with plain decode)

Model	Build	Ran on	Quant	Engine	tok/s	Details
DeepSeek R1 671B	EPYC + single RTX 3090 (hybrid) ⚠ unchecked	RTX 3090	Q2_K	llama.cpp CUDA	8.97	partly in system RAM
DeepSeek V4 Flash	8x RTX 3090 serving DeepSeek V4 Flash (LimeChain vLLM kit)	8x RTX 3090	see source (FP8 KV cache)	vLLM LimeChain SM86 overlay, 1 concurrent request	95.52	📏 Per-session end-to-end p50 at concurrency 1, 128 generated tokens
Flint 40B-A2.5B	RTX 3090 (llm-speed signed run, Oct 2026): Flint 40B-A2.5B Q3_K_XL at 128.9 tok/s	RTX 3090	UD-Q3_K_XL	llama.cpp 0.7.0-dev (build 11803)	128.9
GLM 4.7 Flash	RTX 3090 (llm-speed signed runs, Oct 2026): gpt-oss 20B 207 tok/s, Qwen3.6 27B 43 tok/s	RTX 3090	Q4_K_M (Threadripper PRO host)	Ollama 0.31.1	134.6
Llama 3 70B	Dual RTX 3090 on Ryzen 3700X (X470) ⚠ unchecked	2x RTX 3090	AWQ 4-bit	vLLM (TP=2)	21.2
Llama 3 70B	Dual RTX 3090 on Ryzen 3700X (X470) ⚠ unchecked	2x RTX 3090	Q4_K_M	Ollama	16.95
Llama 3.1 70B	EPYC 7F52 jank rig: 4x3090 + 4090 + 4080 (136 GB VRAM) ⚠ unchecked	mixed / see build	Q8_0	Oobabooga	9
Llama 3.3 70B	Dual used RTX 3090, ~2000 EUR total ⚠ unchecked	2x RTX 3090		ExLlamaV2 (TP)	20
Llama 3.3 70B	2x RTX 3090 (llm-speed signed run, Oct 2026): Llama 3.3 70B Q4_K_M at 19.1 tok/s	2x RTX 3090	Q4_K_M	llama.cpp 0.7.0-dev	19.1
Llama 3.3 70B	RTX 3090 + RTX 4060 Ti 16GB mixed pair ⚠ unchecked	mixed / see build	IQ4_XS		11
Qwen3 235B-A22B	4x RTX 3090 + EPYC 7443, 256 GB DDR4 8-channel ⚠ unchecked	4x RTX 3090	IQ4_XS	llama.cpp	18
Qwen3 235B-A22B	4x RTX 3090 + 32-core Threadripper, 192 GB DDR4 ⚠ unchecked	4x RTX 3090	UD-Q3_K_XL	llama.cpp	12-14
Qwen3.6 27B	RTX 3090 (llm-speed signed runs, Oct 2026): gpt-oss 20B 207 tok/s, Qwen3.6 27B 43 tok/s	RTX 3090	Q4_K_M	llama.cpp 0.7.0-dev	42.7
Qwen3.6 27B	RTX 3090: 2026 models (llamabench community)	RTX 3090	llama-bench tg128 (quant per result)	llama.cpp	40.2
Qwen3.6 35B-A3B	4x RTX 3090 lab: Qwen3.6 on vLLM vs llama.cpp	2x RTX 3090	AWQ-INT4 + MTP-3 (TP=2)	vLLM	225-231	⚡ spec-decode (MTP)
Qwen3.6 35B-A3B	4x RTX 3090 lab: Qwen3.6 on vLLM vs llama.cpp	2x RTX 3090	AWQ-INT4 (TP=2)	vLLM	149
Qwen3.6 35B-A3B	RTX 3090: 2026 models (llamabench community)	RTX 3090	llama-bench tg128 (quant per result)	llama.cpp	146-153
Qwen3.6 35B-A3B	Single RTX 3090: Qwen3.6 35B-A3B (Giles Thomas)	RTX 3090	Q4 GGUF, all layers on GPU, ~90K ctx	llama.cpp CUDA	139.6
Qwen3.6 35B-A3B	4x RTX 3090 lab: Qwen3.6 on vLLM vs llama.cpp	RTX 3090	IQ4_XS (1 GPU)	llama.cpp	115-133
Qwen3.8 27B	Single RTX 3090: Qwen3.8 27B at 78-140 tok/s with the OpenJet megakernel vs llama.cpp	RTX 3090	Q4_K_M (unsloth), greedy code prompt	OpenJet megakernel (MTP, 4 drafts)	139.7	⚡ spec-decode (MTP)
Qwen3.8 27B	Single RTX 3090: Qwen3.8 27B at 78-140 tok/s with the OpenJet megakernel vs llama.cpp	RTX 3090	Q4_K_M (unsloth), reasoning mode	OpenJet megakernel (MTP)	78.0	⚡ spec-decode (MTP)
Qwen3.8 27B	Single RTX 3090: Qwen3.8 27B at 78-140 tok/s with the OpenJet megakernel vs llama.cpp	RTX 3090	Q4_K_M (unsloth), greedy code prompt	llama.cpp (MTP, 2 drafts)	72.5	⚡ spec-decode (MTP)
Qwen3.8 27B	Single RTX 3090: Qwen3.8 27B at 78-140 tok/s with the OpenJet megakernel vs llama.cpp	RTX 3090	Q4_K_M (unsloth), reasoning mode	llama.cpp (MTP, 2 drafts)	50.9	⚡ spec-decode (MTP)
Qwen3.8 27B	Single RTX 3090: Qwen3.8 27B at 78-140 tok/s with the OpenJet megakernel vs llama.cpp	RTX 3090	Q4_K_M (unsloth), no speculation	OpenJet megakernel	42.9
Qwen3.8 27B	Single RTX 3090: Qwen3.8 27B at 78-140 tok/s with the OpenJet megakernel vs llama.cpp	RTX 3090	Q4_K_M (unsloth), no speculation	llama.cpp	40.4
Qwen3.8 27B	RTX 3090: 2026 models (llamabench community)	RTX 3090	llama-bench tg128 (quant per result)	llama.cpp	38.2-71.6
gpt-oss 20B	RTX 3090 (llm-speed signed runs, Oct 2026): gpt-oss 20B 207 tok/s, Qwen3.6 27B 43 tok/s	RTX 3090	reported as Q4_K_M	llama.cpp 0.7.0-dev	207.2
gpt-oss 20B	Single RTX 3090 24GB	RTX 3090	MXFP4	llama.cpp	161.77

All RTX 3090 builds: see also all GPUs · best LLMs by memory size · download the dataset.

FAQ

How fast is the RTX 3090 for local LLMs?

The fastest plain-decode result we have is 207.2 tok/s on gpt-oss 20B (reported as Q4_K_M, llama.cpp 0.7.0-dev) in RTX 3090 (llm-speed signed runs, Oct 2026): gpt-oss 20B 207 tok/s, Qwen3.6 27B 43 tok/s. Speed depends heavily on the model, quant and engine, so compare like with like in the table. RigBase lists 28 measured results from 18 builds.

How much memory does the RTX 3090 have?

24 GB per card. Memory bandwidth: 936 GB/s (memory bandwidth as listed on llamabench).

Which models have been measured on the RTX 3090?

DeepSeek R1 671B, DeepSeek V4 Flash, Flint 40B-A2.5B, GLM 4.7 Flash, Llama 3 70B, Llama 3.1 70B, Llama 3.3 70B, Qwen3 235B-A22B, Qwen3.6 27B, Qwen3.6 35B-A3B, Qwen3.8 27B, gpt-oss 20B.

Where do these numbers come from?

Every row links to a build page with its original public source (benchmark logs, READMEs, forum posts). RigBase does not re-run the benchmarks. Results marked ⚡ use speculative decoding and are not comparable 1:1 with plain decode.

Not enough VRAM? Rent before you buy. Run RTX 3090-class models on a rented 24–96 GB cloud GPU by the hour and see the real speed before spending €1,000+. Try it on Runpod
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.