Skip to content
← CUDA, Triton & GPU Performance / 17
expertNewNVIDIAOpenAITogether AI

On a B200, how does an NVFP4 GEMM apply its block scales, and when does FP4 actually make a layer faster than FP8?

FP4 halves the bytes and doubles the peak, so it must be twice as fast. The answer finds where the 16-element scales are applied, why their memory layout is a silent-wrong-answer trap, and the batch where FP4 stops paying.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

FP4 halves the bytes and doubles the peak, so it must be twice as fast. The answer finds where the 16-element scales are applied, why their memory layout is a silent-wrong-answer trap, and the batch where FP4 stops paying.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackA cp.async double-buffered kernel fails a handful of CI runs and never fails under -G. Find the race and fix it.Next in this trackA kernel PR claims 12x over eager PyTorch on the SwiGLU activation and passes the tests. How do you decide whether it is real?Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.