Absorption is one line of matrix associativity. The part interviewers score is the bill: attention gets more expensive per cached token, and the kernel lands in a roofline regime GQA never visits.
an account raises the free count per topic
← KV Cache & GPU Memory / 09
expertNewGoogle DeepMindNVIDIAFireworks AI
Why can MLA decode attend over the 512-dimensional latent without decompressing keys, and what does absorption cost in FLOPs?
Absorption is one line of matrix associativity. The part interviewers score is the bill: attention gets more expensive per cached token, and the kernel lands in a roofline regime GQA never visits.
Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.
UP NEXT ON YOUR JOURNEY
Next in this trackWith parallel sampling at n=8, a 2K-token prompt and 500 output tokens each, how many KV blocks does copy-on-write save?Next in this trackThe KV pool is full and requests keep arriving. Compare admission control, preemption and spilling, and what each does to p99.Next in this trackIn prefill-decode disaggregation, how does KV transfer size scale, and what link bandwidth keeps the transfer off the critical path?
DISCUSSION · 0
Nobody has commented on this answer. Add a correction or another way to work it.