Skip to content
← Quantization & Numerics / 18
hardNewNVIDIAGoogle DeepMindMeta

NVFP4 scales each block's max to 6. Why can scaling some blocks to 4 lower the error, and how do you choose per block?

FP4 has no value between 4 and 6, so the largest numbers in a block lose the most. The answer derives the error at the top of the grid for both scale targets, works the selection rule on two blocks, explains the tensor-scale change it forces, and says why MXFP4 cannot do it and where it made results worse.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

FP4 has no value between 4 and 6, so the largest numbers in a block lose the most. The answer derives the error at the top of the grid for both scale targets, works the selection rule on two blocks, explains the tensor-scale change it forces, and says why MXFP4 cannot do it and where it made results worse.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackMoving rollouts and trainer from BF16 to FP16 shrinks their logprob gap about 8x; deterministic mode does nothing. Why?Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?Popular right nowDecode attention intensity is fixed at the query-to-KV head ratio. Derive that, then tell me what MLA does to it.
EDITOR'S NOTE

Strong candidates find the 4-to-6 gap themselves before naming the method. The follow-up that separates them is why the tensor scale has to drop from 448 to 256.

DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.