mla

Batching rescues the weight GEMMs at decode but never the attention kernel, and one head-count ratio explains why. The answer derives it, then shows how MLA moves attention to within a hair of the H100 ridge.

The formula takes one line. The answer interviewers score is the memory budget around it: what the engine reserves first, what tensor parallelism does to the heads, and why the count you get is a ceiling.

Absorption is one line of matrix associativity. The part interviewers score is the bill: attention gets more expensive per cached token, and the kernel lands in a roofline regime GQA never visits.

The per-token formula gives a scary number for GQA and a reassuring one for MLA. Put both on real GPUs with tensor parallelism and the reassuring one is the request that does not fit.

Tensor parallelism splits a GQA cache by KV head. MLA has no KV heads to split, so TP copies the whole cache onto every GPU without any warning. The answer measures that waste on an H200 node and prices the fix.