Skip to content

bf16

Inference interview questions tagged bf16, across every topic.

2 questions · 0 unlocked for you
Open full answer →
Open full answer →

Concepts behind "bf16"

Concept pages for the mechanisms these questions lean on.

Foundational
🔢 Quantization & Numerics
Number Formats for Inference: BF16, FP8, INT4, FP4Inference formats differ in three ways that matter: bytes per element, which sets decode speed and memory; exponent bits versus mantissa bits, which sets range and precision; and whether the GPU has tensor cores that multiply them natively. Llama 3.3 70B is 141 GB in BF16, 71 GB in FP8 and about 40 GB in NVFP4.
Explore the full curriculum →