Decode attention intensity is fixed at the query-to-KV head ratio. Derive that, then tell me what MLA does to it.

TL;DR: Each cached K or V element is read once per step and used by the $g = n_h / n_{kv}$ query heads that share it, so decode attention runs at $2g/b$ FLOP/byte: 8 for Llama 3.3 70B in BF16, whatever the batch. MLA lets all 128 DeepSeek V3 heads attend over one 576-wide latent, which lifts it to about 242 FLOP/byte, close to the H100's 295 ridge, until tensor parallelism splits the heads and divides it back down.

How to approach it

Count bytes and FLOPs for one cached token in one layer, because that is the unit attention repeats over. Ask what the batch is (the trap is that it does not matter), how many query tokens per sequence each step carries (speculative verification changes the answer), the KV precision, and how the heads are sharded across GPUs. Then compare the result to the ridge of the part.

A strong answer

At decode each sequence brings one new query token per head and its own cache. For one cached token in one layer, a KV head stores $d_h$ key and $d_h$ value elements, and each of the $g$ query heads mapped to it does a $d_h$-long dot product for the score and a $d_h$-long multiply-add for the value sum:

$$I_{\text{attn}} = \frac{g \cdot (2 d_h + 2 d_h)}{2 d_h \cdot b} = \frac{2g}{b}$$

Context length cancels, because FLOPs and bytes both grow with $T$. Batch cancels, because sequence $i$ never reads sequence $j$'s cache: 64 sequences mean 64 times the bytes and 64 times the FLOPs. This is the opposite of the weight GEMMs, where one weight read serves every sequence in the batch.

Llama 3.3 70B: 64 query heads / 8 KV heads    -> g = 8
per layer, per KV head, T = 8,192, BF16:
  bytes = 2 x 8,192 x 128 x 2 B            = 4.19 MB
  FLOPs = 4 x 8 x 8,192 x 128              = 33.6 MFLOP
  I     = 33.6e6 / 4.19e6                  = 8 FLOP/byte  (= 2g/b)
H100 SXM ridge = 989 / 3.35                = 295  ->  attention 37x below it
FP8 KV cache: b = 1                        -> 16 FLOP/byte, still bandwidth-bound

MLA changes the structure, not the formula. DeepSeek V3 caches one latent of 512 plus a decoupled RoPE key of width 64, per token and per layer, shared by all 128 heads. With the key up-projection absorbed into the query and the value up-projection absorbed into the output, each head scores against the full 576 elements and aggregates over the 512 latent elements:

bytes per cached token per layer = (512 + 64) x 2 B            = 1,152 B
FLOPs per cached token per layer = 128 x (2 x 576 + 2 x 512)   = 278,528
I (BF16 latent)                  = 278,528 / 1,152             = 242 FLOP/byte
I (FP8 latent)                   = 278,528 / 576               = 484 FLOP/byte
sanity: 242 / 8 = 30x the GQA kernel's intensity from the same formula,
        with g = 128 heads sharing one cache line and wider dot products

The last row is the second-order effect most candidates miss. Under tensor parallelism, GQA shards KV heads, so each GPU reads only its share and intensity is unchanged. MLA has a single latent that every head needs, so TP=8 replicates the latent on every GPU while each GPU holds 16 of the 128 heads: the bytes stay, the FLOPs drop 8x, and intensity falls to 30. That is why MLA deployments run attention data-parallel (each GPU owns whole sequences with all 128 heads) and shard only the MoE layers, as SGLang's enable_dp_attention option does (opt-in in v0.5.21, checked October 2026).

Absorption is also what makes the number possible at all. Without it, decode must decompress each cached latent into per-head keys and values every step, $2 \times 512 \times 128 \times 256 \approx 33.6$ MFLOP per cached token per layer, 120 times the absorbed cost. Caching the decompressed keys and values instead would cost 5.0 MB per token across 61 layers and throw away the memory win.

The reversal condition: intensity is fixed per query token, so anything that puts several query tokens against one cache read multiplies it. Speculative verification of 3 draft tokens plus the bonus position gives 4 queries per sequence: GQA rises to 32 and absorbed MLA to about 970, past the BF16 ridge, so MLA verification becomes compute-bound. Shared-prefix kernels that read a common system prompt once for many sequences do the same for the prefix portion.

What interviewers probe next

"Then why does batching help decode throughput at all?" Because the weight GEMMs dominate bytes at short context and their intensity is $2B/b$. Attention's share of the step grows with batch times context, and once KV bytes exceed weight bytes, more batch stops helping.

"Does FP8 KV make MLA decode compute-bound on an H100?" At 484 FLOP/byte against a BF16-math ridge of 295, yes, provided the kernel dequantizes in registers. Against an FP8-math ridge of 591 it would still sit just below.

"Why not raise the GQA ratio to 64 and get MQA's intensity?" Quality and TP. One KV head cannot be split across 8 GPUs, so it replicates, which is the MLA problem with none of MLA's expressiveness.

"What does a B200 do to the GQA kernel?" Nothing to intensity. The ridge drops slightly to 281, and the kernel gets 2.4x faster because bandwidth rises from 3.35 to 8 TB/s.

How it's scored

Weak: says batching raises attention intensity, or computes attention FLOPs with KV heads and gets intensity 1.

Hire: derives $2g/b$, shows batch and context cancel, gets 8 FLOP/byte for Llama 70B against a 295 ridge, and computes MLA's 242 from the 576-wide latent.

Strong Hire: explains that absorption is what avoids per-step decompression, shows TP replicates the latent and divides MLA intensity, and identifies multi-token verification as the case that pushes MLA attention past the ridge.

Common mistakes

• Treating MLA's latent as a 512-wide head with $g = 128$ and forgetting the 64-wide RoPE key, which shifts the result by 11%.
• Assuming TP helps MLA attention the way it helps GQA. It replicates the latent and shrinks per-GPU FLOPs.
• Stating that FP8 KV makes GQA attention compute-bound. At 16 FLOP/byte it is still 18x below the ridge, only twice as fast.
• Quoting attention as "memory-bound" without the bytes: at batch 32 and 8K context on four tensor-parallel H100s, Llama 70B reads 21.5 GB of KV per GPU, about 6.4 ms of HBM time per step.