fp4
Inference interview questions tagged fp4, across every topic.
2 questions · 0 unlocked for you
Concepts behind "fp4"
Concept pages for the mechanisms these questions lean on.
Foundational
Number Formats for Inference: BF16, FP8, INT4, FP4Inference formats differ in three ways that matter: bytes per element, which sets decode speed and memory; exponent bits versus mantissa bits, which sets range and precision; and whether the GPU has tensor cores that multiply them natively. Llama 3.3 70B is 141 GB in BF16, 71 GB in FP8 and about 40 GB in NVFP4.🔢 Quantization & Numerics
Advanced
Block-Scaled FP4: MXFP4 and NVFP4MXFP4 and NVFP4 both store E2M1 4-bit values with a shared scale per block. MXFP4 uses one power-of-two E8M0 scale per 32 elements (4.25 bits); NVFP4 uses one E4M3 scale per 16 plus an FP32 per-tensor scale (4.5 bits). The scale format decides how much of FP4's tiny grid each block actually uses.🔢 Quantization & Numerics🔒 Premium