Best local LLMs for 128 GB or more of VRAM or unified memory

Measured speeds from builds where the model ran on more than 96 GB of GPU or unified memory. Each number links to its build and source.

8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 64 GB · 96 GB · 128 GB or more

GPUs in this tier: DGX Spark / GB10

Fastest measured result per model

Excludes results the source marks pending and results where part of the model sat in system RAM.

All results in this tier

Hide speculative-decoding results ⚡ (MTP, DFlash, DSpark and similar draft-and-verify methods give much higher tok/s on predictable text and are not comparable 1:1 with plain decode)

What fits by memory math

Q4_K_M weights (4.8944 bits per weight, per the llama.cpp quantize table) that fit in 128 GB with ~5% headroom, before KV cache: Qwen3.8 Flash-Next ≈110 GB, Mistral Medium 3.5 128B ≈78 GB, Ornith 1.5 122B ≈77 GB, Qwen3.5 122B ≈75 GB, Nemotron 3 Super 120B-A12B ≈73 GB, Nemotron 3.5 Super VL 120B-A12B ≈73 GB, gpt-oss 120B ≈72 GB, Nemotron 3.5 Bolt 74B-A9B ≈45 GB, Llama 3.3 70B ≈43 GB, Flint 40B-A2.5B ≈24 GB, Ornith 1.5 35B-A3B ≈22 GB, Qwen3.6 35B-A3B ≈22 GB, Gemma 4 31B ≈19 GB, Nemotron 3.5 Lightning 30B-A3B ≈18 GB, Qwen3.6 27B ≈17 GB, Qwen3.8 27B ≈17 GB, Gemma 4 26B-A4B ≈15 GB, Devstral Small 2 24B ≈15 GB, LFM2.5 24B-A2B ≈15 GB, gpt-oss 20B ≈13 GB, Gemma 4 12B ≈7 GB, Qwen3.5 9B ≈5 GB, Llama 3.1 8B ≈5 GB.

FAQ

Which local LLMs have been measured on 128 GB or more of VRAM?

Llama 2 7B, gpt-oss 20B, gpt-oss 120B, DeepSeek R1 671B, Llama 3.1 70B, Llama 3.1 405B, DeepSeek R1 Distill 14B, Llama 3.2 3B, Muse Spark Open 816B-A42B, Qwen3.8 27B, Mistral Medium 3.5 128B, Qwen3.6 35B-A3B, Llama 3.1 8B, Nemotron 3.5 Super VL 120B-A12B, Nemotron 3.5 Bolt 74B-A9B, Nemotron 3.5 Lightning 30B-A3B, Nemotron 3 Nano Omni 30B, Gemma 4 26B-A4B, Qwen3.8 Flash-Next, Nemotron 3 Super 120B-A12B, MiniMax M2.7, DeepSeek V4 Flash, Qwen3.6 27B, Gemma 4 31B, GLM-5.2, GLM-5.3 Flash (87 results).

What fits in 128 GB or more of VRAM?

By memory math at Q4_K_M (4.8944 bits per weight, llama.cpp), weights only, before KV cache and context: Qwen3.8 Flash-Next (≈110 GB), Mistral Medium 3.5 128B (≈78 GB), Ornith 1.5 122B (≈77 GB), Qwen3.5 122B (≈75 GB), Nemotron 3 Super 120B-A12B (≈73 GB), Nemotron 3.5 Super VL 120B-A12B (≈73 GB), gpt-oss 120B (≈72 GB), Nemotron 3.5 Bolt 74B-A9B (≈45 GB). Leave a few GB for context.

Are speculative-decoding results comparable?

Not 1:1. MTP, DFlash and similar methods draft tokens that the main model verifies, which can double tok/s on predictable text. This page lists them in a separate column; tick the box to hide them in the full table.