Skip to content
← Implement-from-Scratch Coding / 11
hardNewNVIDIAMetaGoogle DeepMind

Implement beam search over a paged KV cache, forking beams by sharing blocks and copying only the tail on write.

Beam search reshuffles which hypotheses survive every step, and a naive cache copies every survivor's history. The answer forks block tables instead, gets the release order right so the last sharer writes in place, and proves it against a beam search that never pages.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Beam search reshuffles which hypotheses survive every step, and a naive cache copies every survivor's history. The answer forks block tables instead, gets the release order right so the last sharer writes in place, and proves it against a beam search that never pages.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackImplement a per-tenant rate limiter for an LLM API that limits tokens, not requests, using a token bucket.Next in this trackImplement a KV block allocator that shares prefix blocks across requests by chained hashes, with LRU eviction of cached free blocks.Next in this trackImplement a minimal autoregressive generate loop that streams text and honors stop sequences, including ones split across tokens.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.