Skip to content
← Speculative & Constrained Decoding / 12
mediumNewGoogle DeepMindMetaNVIDIA

Implement beam search on a paged KV cache. How do forking and pruning work on block tables, and what does beam width 4 cost in memory?

Four beams over a 1,000-token prompt cost 600 MiB if each keeps its own cache and as little as 156 MiB if they share blocks. The answer implements the fork, the copy-on-write and the prune, then shows what vLLM actually does today.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Four beams over a 1,000-token prompt cost 600 MiB if each keeps its own cache and as little as 156 MiB if they share blocks. The answer implements the fork, the copy-on-write and the prune, then shows what vLLM actually does today.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackSpeculative decoding with a quantized target or a quantized draft: does losslessness still hold, and against which distribution?Next in this trackChoose the speculative draft length per request at runtime. What signal, what policy, and how much does it beat a fixed gamma?Next in this trackDesign speculative decoding for a 70B fleet whose batch size swings from 1 at night to 256 at peak. What runs at each load level?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.