RTX A4000 for local LLMs

3 measured results from 2 builds, each linked to its source.

Memory	16 GB per card
Memory bandwidth	not listed
Typical EU price	≈€1,050 new (PNY RTX A4000 16 GB GDDR6 VCNRTXA4000-PB, billiger.de, 2026-10-24)

Can it run? Muse Glimmer 30B on RTX A4000 gpt-oss 20B on RTX A4000

All builds with the RTX A4000

Build	GPU(s)	Memory GB (total)	Results
Intel Core Ultra 265K + RTX A4000	RTX A4000	16	2
RTX A4000 16GB (llm-speed signed run, Oct 2026): Muse Glimmer 30B UD-IQ3_XXS at 21.4 tok/s	RTX A4000	16	1

All measured results

Hide speculative-decoding results ⚡ (MTP, DFlash, DSpark and similar draft-and-verify methods give much higher tok/s on predictable text and are not comparable 1:1 with plain decode)

Model	Build	Ran on	Quant	Engine	tok/s
Muse Glimmer 30B	RTX A4000 16GB (llm-speed signed run, Oct 2026): Muse Glimmer 30B UD-IQ3_XXS at 21.4 tok/s	RTX A4000	UD-IQ3_XXS	llama.cpp 0.7.0-dev (build 11803)	21.42
gpt-oss 20B	Intel Core Ultra 265K + RTX A4000	RTX A4000		llama.cpp	93.55
Llama 3.2 3B legacy	Intel Core Ultra 265K + RTX A4000	RTX A4000		llama.cpp	149.44

All RTX A4000 builds: see also all GPUs · best LLMs by memory size · download the dataset.

FAQ

How fast is the RTX A4000 for local LLMs?

The fastest plain-decode result we have is 149.44 tok/s on Llama 3.2 3B (, llama.cpp) in Intel Core Ultra 265K + RTX A4000. Speed depends heavily on the model, quant and engine, so compare like with like in the table. RigBase lists 3 measured results from 2 builds.

How much memory does the RTX A4000 have?

16 GB per card.

Which models have been measured on the RTX A4000?

Llama 3.2 3B, Muse Glimmer 30B, gpt-oss 20B.

Where do these numbers come from?

Every row links to a build page with its original public source (benchmark logs, READMEs, forum posts). RigBase does not re-run the benchmarks. Results marked ⚡ use speculative decoding and are not comparable 1:1 with plain decode.

Not enough VRAM? Rent before you buy. Run RTX A4000-class models on a rented 24–96 GB cloud GPU by the hour and see the real speed before spending €1,000+. Try it on Runpod
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.