Skip to content
← KV Cache & GPU Memory / 17
hardNewNVIDIATogether AIFireworks AI

DeepSeek-V4 compresses its KV cache per layer. Size it for a 1M-token request and say which bytes a decode step reads.

The per-token formula gives 65 GB for a 1M-token DeepSeek-V4 request. The real number is 11x smaller, and the bytes that dominate a decode step are not the ones that dominate the footprint.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

The per-token formula gives 65 GB for a 1M-token DeepSeek-V4 request. The real number is 11x smaller, and the bytes that dominate a decode step are not the ones that dominate the footprint.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackDesign the contract between an engine's scheduler and an external KV store. Who owns a block while it loads or saves?Next in this trackAfter enabling CPU KV offload, requests that hit the host tier have worse TTFT than requests that miss. Find out why.Next in this trackDeepSeek says V4.1-Flash keeps 890 bytes of KV per token. vLLM allocates 1,056. Reconcile the two numbers.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.