Skip to content
← Inference Physics & Roofline / 07
hardNewOpenAIGoogle DeepMindxAI

DeepSeek V3 has 671B parameters but only 37B active. Why does a decode step at batch 64 read far more than 37B?

Active parameters describe one token. A decode step serves a batch, and the batch decides how many experts get pulled through HBM. The answer computes that number and shows where the MoE cost advantage actually comes from.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Active parameters describe one token. A decode step serves a batch, and the batch decides how many experts get pulled through HBM. The answer computes that number and shows where the MoE cost advantage actually comes from.

an account raises the free count per topic
UP NEXT ON YOUR JOURNEY
Next in this trackHow would you measure MBU on a running server? What do you time, what do you count, and which numbers make the result wrong?Next in this trackYour decode step runs at 60% of the bandwidth floor. Where does the rest of the time go? Build me the budget.Next in this trackLong context: at what sequence length do KV reads overtake weight reads for a 70B GQA model at batch 16?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.