Run Bigger AI Models on a Smaller GPU with 64GB DDR5

November 16, 2026
Published on  Updated on  
Local AI · DDR5 Guide

Run Bigger AI Models on a Smaller GPU with 64GB DDR5

Mixture-of-Experts models are the biggest change in local AI this year. They let system memory carry most of the model, so a mid-range graphics card plus plenty of DDR5 can run models that used to need a workstation GPU.

Quick Answer

Mixture-of-Experts (MoE) models only use a small part of their parameters for each token, so most of the model can live in system RAM instead of GPU VRAM. With an 8–16GB graphics card and 64GB of DDR5, 30B–35B MoE models run comfortably. With 128GB, 100B-class MoE models become possible. Capacity decides what fits; dual-channel bandwidth decides how fast it runs.

Figures are estimates for llama.cpp-based tools at 4-bit (Q4) quantization. Speeds vary by CPU, GPU, memory speed and software version.
~3B
Active Params in a 35B-A3B Model
64 GB
Recommended DDR5 for MoE
~96 GB/s
Dual-Channel DDR5-6000 Peak

What Is a Mixture-of-Experts Model?

A traditional "dense" model uses every parameter for every token it generates. A Mixture-of-Experts model splits much of its network into many smaller "experts," and a router picks only a few of them for each token.

You can usually spot an MoE model from its name. Qwen 3.6 35B-A3B has 35 billion parameters in total, and "A3B" means about 3 billion are active per token. Other popular examples include Gemma 4 26B-A4B and gpt-oss 20B (about 21 billion total, 3.6 billion active).

Dense Model MoE Model
Memory needed All parameters All parameters (same as dense)
Compute per token All parameters Only the active experts
Runs well partly from system RAM? Slows down a lot Yes, much better

Why MoE Makes System RAM So Valuable

The catch with MoE is that the whole model still has to sit somewhere in memory. A 30B-class MoE model at Q4 is about 20GB, which is too big for most 8–16GB graphics cards.

The solution is expert offloading. llama.cpp, the engine behind many popular local AI apps, can keep the parts used on every token, such as attention layers and shared weights, on the GPU, and leave the expert weights in system RAM. Because only a few experts are used per token, the CPU reads only a small slice of the model each time, and the model stays responsive.

llama.cpp example
llama-server -m your-moe-model-Q4_K_M.gguf -ngl 99 --n-cpu-moe 24 -c 16384
--n-cpu-moe N keeps the expert weights of N layers in system RAM. Raise N until the model fits in VRAM, then lower it as far as your VRAM allows. --cpu-moe moves all experts to system RAM.

How Much DDR5 Do You Need?

Your GPU System RAM What You Can Run (Q4)
No GPU / integrated graphics 32 GB Small MoE models such as gpt-oss 20B, at modest speed
8 GB VRAM 64 GB 30B–35B MoE models with expert offload
12–16 GB VRAM 64 GB 30B–35B MoE models with long context, or two models at once
16–24 GB VRAM 128 GB (4×32GB) 100B-class MoE models such as gpt-oss 120B (~65GB) or Qwen 3.5 122B (~81GB)
Leave headroom. Your operating system, apps and the KV cache need memory too. A 65GB model will not run well on a 64GB system, but runs comfortably on 128GB. Read more in How Context Length and KV Cache Affect RAM.

Capacity First, Then Bandwidth

When expert weights live in system RAM, every generated token reads part of them from DDR5. That makes memory bandwidth the speed limit.

  • Always run dual-channel. Two matched modules double the bandwidth of one. DDR5-6000 in dual-channel delivers about 96GB/s in theory (6,000 MT/s × 8 bytes × 2 channels).
  • Higher MT/s helps. Moving from DDR5-4800 to DDR5-6000 raises peak bandwidth by about 25%.
  • Tighter timings are a bonus, not the priority. See A Complete Guide to Memory Timings.

Recommended Silicon Power DDR5 Kits for MoE Workloads

Silicon Power DDR5 · 64GB (2×32GB) Kits
Top Pick

Zenith RGB DDR5-6000 CL30 64GB (2×32GB)

Low-latency 6000 MT/s for the fastest offloaded token generation, with AMD EXPO and Intel XMP 3.0.

DDR5-6000CL30RGB
View product →
No RGB

Value Gaming DDR5-6000 CL30 64GB (2×32GB)

The same speed and latency in a clean, non-RGB design.

DDR5-6000CL30
View product →
Budget

Pulse Gaming DDR5-6000 CL38 64GB (2×32GB)

The most affordable way to reach 64GB at 6000 MT/s.

DDR5-6000CL38
View product →
Fast Storage

US75 NVMe SSD

MoE models are large downloads. A fast Gen4 drive loads a 20GB model in a few seconds.

PCIe Gen4Up to 7,000 MB/s
View product →
Going to 128GB: Silicon Power DDR5 modules go up to 32GB each, so 128GB uses two 64GB (2×32GB) kits on a four-slot motherboard. Many platforms run four DDR5 modules at reduced speed, so check your motherboard's QVL. For a big library of 65GB+ models, add a UD90 4TB.

Dense or MoE: Which Should You Choose?

  • Your GPU has enough VRAM for a dense model: a dense 27B model is simple to run and very consistent on long documents.
  • Your GPU is small but you have plenty of RAM: an MoE model with expert offload is usually the strongest option.
  • You want to try very large models: MoE is the only practical route on a desktop PC.
Key Takeaways
  • MoE models need memory for all parameters but compute only a few billion per token.
  • Expert offloading lets DDR5 hold most of the model while the GPU handles the rest.
  • 64GB (2×32GB) DDR5 is the sweet spot; 128GB opens up 100B-class MoE models.
  • Dual-channel and higher MT/s directly increase offloaded generation speed.

Frequently Asked Questions

Do MoE models need less memory than dense models?

No. They need memory for all their parameters. What they save is compute per token, which is why they still run quickly when the expert weights sit in system RAM.

Is DDR4 good enough for MoE offloading?

It works, but DDR4 has much less bandwidth than DDR5, so offloaded models generate tokens more slowly. If you are building a new PC for local AI, choose DDR5.

Should I buy 2×32GB or 4×16GB?

2×32GB. It gives you 64GB in two slots, usually runs at higher speed, and leaves two slots free. If you later want 128GB, buying the same kit again gives you the best chance of stable operation.

How do I know if a model is MoE?

Look for a name like "35B-A3B" or "26B-A4B". The first number is total parameters and the "A" number is the active parameters per token. The model card on Ollama or Hugging Face will also say "Mixture of Experts".

Local AIMoEDDR564GB RAMllama.cppCPU Offload

← Back to the Local AI Hardware Guide

Published on  Updated on