Best local LLMs for 64 GB of VRAM or unified memory
Measured speeds from builds where the model ran on more than 48 and up to 64 GB of GPU or unified memory. Each number links to its build and source.
8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 64 GB · 96 GB · 128 GB or more
Fastest measured result per model
| Model | Plain decode tok/s | With speculative decoding ⚡ | Results |
|---|---|---|---|
| Qwen3.8 27B | 42.4 (Q8_0, 2x Tesla V100 32GB) | 104 (Q8_0, 2x Tesla V100 32GB) | 3 |
| Qwen3.8 Flash-Next | – | – | 3 |
| Llama 3 8B | 50.74 (Q4_K_M, Apple M3 Max) | – | 2 |
| Llama 3.3 70B | 11 (Q4, Apple unified memory) | – | 1 |
| gpt-oss 20B | 75.15 (MXFP4, Apple M1 Max) | – | 1 |
| DeepSeek V4.1 Flash | – | – | 1 |
Excludes results the source marks pending and results where part of the model sat in system RAM.
All results in this tier
(MTP, DFlash, DSpark and similar draft-and-verify methods give much higher tok/s on predictable text and are not comparable 1:1 with plain decode)
| Model | Build | Ran on | Quant | Engine | tok/s | Details |
|---|---|---|---|---|---|---|
| DeepSeek V4.1 Flash | Dual Tesla V100 32 GB on EPYC 7402 (bnk engine) | 2x Tesla V100 32GB (64 GB) | GSQ-RCO 2-bit | bnk v0.5.0, experts split VRAM/RAM (CPU offload) | 13 | 📏 Maintainer decode tok/s from the bnk READMEpartly in system RAM |
| Llama 3 8B | MacBook Pro M3 Max 40-core GPU, 64 GB | Apple M3 Max (64 GB) | Q4_K_M | llama.cpp | 50.74 | |
| Llama 3 8B | MacBook Pro M1 Max 32-core GPU, 64 GB | Apple M1 Max (64 GB) | Q4_K_M | llama.cpp | 34.49 | |
| Llama 3.3 70B | Mac Studio 64 GB ⚠ unchecked | Apple unified memory (64 GB) | Q4 | 11 | ||
| Qwen3.8 27B | Dual Tesla V100 32 GB on EPYC 7402 (bnk engine) | 2x Tesla V100 32GB (64 GB) | Q8_0 | bnk dense engine, tensor parallel, MTP drafting | 104 | ⚡ spec-decode (MTP)📏 Maintainer decode tok/s from the bnk README |
| Qwen3.8 27B | Dual Intel Arc Pro B70 (neural.download lab) | 2x Intel Arc Pro B70 (64 GB) | FP8 | vLLM XPU, TP2, MTP 5 | 46.01 | ⚡ spec-decode (MTP)📏 Lab README/scoreboard |
| Qwen3.8 27B | Dual Tesla V100 32 GB on EPYC 7402 (bnk engine) | 2x Tesla V100 32GB (64 GB) | Q8_0 | bnk dense engine, tensor parallel, no drafting | 42.4 | 📏 Maintainer decode tok/s from the bnk README |
| Qwen3.8 Flash-Next | Dual Tesla V100 32 GB on EPYC 7402 (bnk engine) | 2x Tesla V100 32GB (64 GB) | IQ3_S | bnk v0.3.1, decode at 4K context | 121 | context: 4K📏 Maintainer decode tok/s from the bnk READMEpartly in system RAM |
| Qwen3.8 Flash-Next | Dual Tesla V100 32 GB on EPYC 7402 (bnk engine) | 2x Tesla V100 32GB (64 GB) | FP4-Local | bnk v0.3.1, decode at 4K context | 114 | context: 4K📏 Maintainer decode tok/s from the bnk READMEpartly in system RAM |
| Qwen3.8 Flash-Next | Dual Intel Arc Pro B70 (neural.download lab) | 2x Intel Arc Pro B70 (64 GB) | IQ3 | neural.download runtime, TP2 | 58.56 | 📏 Lab README/scoreboardpartly in system RAM |
| gpt-oss 20B | Apple M1 Max 64GB | Apple M1 Max (64 GB) | MXFP4 | llama.cpp | 75.15 |
What fits by memory math
Q4_K_M weights (4.8944 bits per weight, per the llama.cpp quantize table) that fit in 64 GB with ~5% headroom, before KV cache: Nemotron 3.5 Bolt 74B-A9B ≈45 GB, Llama 3.3 70B ≈43 GB, Flint 40B-A2.5B ≈24 GB, Ornith 1.5 35B-A3B ≈22 GB, Qwen3.6 35B-A3B ≈22 GB, Gemma 4 31B ≈19 GB, Nemotron 3.5 Lightning 30B-A3B ≈18 GB, Qwen3.6 27B ≈17 GB, Qwen3.8 27B ≈17 GB, Gemma 4 26B-A4B ≈15 GB, Devstral Small 2 24B ≈15 GB, LFM2.5 24B-A2B ≈15 GB, gpt-oss 20B ≈13 GB, Gemma 4 12B ≈7 GB, Qwen3.5 9B ≈5 GB, Llama 3.1 8B ≈5 GB.
FAQ
Which local LLMs have been measured on 64 GB of VRAM?
Llama 3.3 70B, Llama 3 8B, gpt-oss 20B, Qwen3.8 27B, Qwen3.8 Flash-Next, DeepSeek V4.1 Flash (11 results).
What fits in 64 GB of VRAM?
By memory math at Q4_K_M (4.8944 bits per weight, llama.cpp), weights only, before KV cache and context: Nemotron 3.5 Bolt 74B-A9B (≈45 GB), Llama 3.3 70B (≈43 GB), Flint 40B-A2.5B (≈24 GB), Ornith 1.5 35B-A3B (≈22 GB), Qwen3.6 35B-A3B (≈22 GB), Gemma 4 31B (≈19 GB), Nemotron 3.5 Lightning 30B-A3B (≈18 GB), Qwen3.6 27B (≈17 GB). Leave a few GB for context.
Are speculative-decoding results comparable?
Not 1:1. MTP, DFlash and similar methods draft tokens that the main model verifies, which can double tok/s on predictable text. This page lists them in a separate column; tick the box to hide them in the full table.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.