Skip to content
← KV Cache & GPU Memory / 14
expertNewGoogle DeepMindMetaNVIDIA

When you quantize the KV cache, why are keys more sensitive than values, and what per-channel or per-token scheme follows from that?

Keys and values take the same bytes, yet their errors reach the output by different roads. Trace both roads and the scale layout for each tensor follows, along with the bookkeeping that makes per-channel keys awkward to serve.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Keys and values take the same bytes, yet their errors reach the output by different roads. Trace both roads and the scale layout for each tensor follows, along with the bookkeeping that makes per-channel keys awkward to serve.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackDesign a multi-tier KV cache across HBM, host DRAM and SSD for a fleet with heavy prefix reuse, and show the hit-rate math.Next in this trackA hybrid model mixes attention and recurrent layers. Budget its state and design safe prefix reuse and rollback.Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.