Advanced

Block-Scaled FP4: MXFP4 and NVFP4

MXFP4 and NVFP4 both store E2M1 4-bit values with a shared scale per block. MXFP4 uses one power-of-two E8M0 scale per 32 elements (4.25 bits); NVFP4 uses one E4M3 scale per 16 plus an FP32 per-tensor scale (4.5 bits). The scale format decides how much of FP4's tiny grid each block actually uses.

Get full Premium access · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

RELATED CONCEPTS

Number Formats for Inference: BF16, FP8, INT4, FP4 Weight-Only Quantization Versus W8A8 The Roofline Model for LLM Inference The KV Cache: Bytes per Token and the Memory Budget The GPU Memory Hierarchy in Numbers

PRACTICE THIS IN REAL QUESTIONS

Quantization & NumericsAWQ INT4 beat FP8 at batch 1, lost to it at batch 256, and cost 4 points on a coding eval. Explain both and pick a format.→Quantization & NumericsPer-tensor, per-channel, per-group, per-block: where does each quantization scale live, and what does it cost in bytes and in the kernel?→KV Cache & GPU MemoryFP8 KV cache: what scales does it need, where is the error largest, and how would you validate it on long-context tasks?→Quantization & NumericsGPTQ and AWQ both produce INT4 weights with group scales. What does each one optimize, and when does one beat the other?→Performance Debugging & BenchmarkingThroughput fell 15% after you enabled FP8. Find out whether it is the kernel, the scales or the config.→CUDA, Triton & GPU PerformanceA GEMM launches 140 output tiles on a 132-SM H100. What does that cost, and how do you fix it?→