← 🧮 KV Cache & Memory
Advanced
Paged KV Memory and Block Tables
Paged KV memory splits the cache into fixed-size blocks and gives each sequence a block table that maps logical positions to physical blocks, as virtual memory does with pages. It bounds waste to one partial block per sequence, which decides how many sequences fit and makes prefix sharing and copy-on-write possible.
Get full Premium access · ₹2,000 / $25
Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.
6 months · One payment · No auto-renewal
RELATED CONCEPTS
LESSONS THAT TEACH THIS
Serving under pressure: from admission to releaseReserve the page a request has not written yet12 min→Reading inference engines: prove what the code preservesFollow one token through five ownership boundaries10 min→Reading inference engines: prove what the code preservesProve cached execution against the full prefix12 min→
PRACTICE THIS IN REAL QUESTIONS
KV Cache & GPU MemoryDerive the KV cache footprint for MHA, GQA and MLA, then tell me how many 8K-token conversations fit on four H100s.→Serving Engine InternalsWalk one request through vLLM V1, from the HTTP call to the streamed token. Which process owns each step, and why is it split that way?→Performance Debugging & BenchmarkingAfter a prompt-template deploy, p99 TTFT went from about 100 ms to over a second while GPU utilization did not move. Debug it.→Implement-from-Scratch CodingImplement a paged KV-cache block manager with copy-on-write forking, then a scheduler step that preempts when blocks run out.→KV Cache & GPU MemoryWhy does a paged KV cache waste at most one block per sequence, and how would you pick the block size?→Attention KernelsWrite the causal mask for a decode step with a KV cache, where the query is shorter than the keys. Which corner does it align to?→