Skip to content
← Implement-from-Scratch Coding / 13
hardNewAnthropicOpenAIPerplexity

Implement a KV block allocator that shares prefix blocks across requests by chained hashes, with LRU eviction of cached free blocks.

Two prompts with the same 16 tokens in block 3 must not share block 3 unless blocks 0 to 2 match too. The answer chains the hashes, keeps freed blocks hittable until memory is needed, evicts tails before heads, and proves a colliding hash cannot hand a request the wrong KV.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Two prompts with the same 16 tokens in block 3 must not share block 3 unless blocks 0 to 2 match too. The answer chains the hashes, keeps freed blocks hittable until memory is needed, evicts tails before heads, and proves a colliding hash cannot hand a request the wrong KV.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackImplement a minimal autoregressive generate loop that streams text and honors stop sequences, including ones split across tokens.Next in this trackImplement split-KV decoding attention and the merge of the partial results, and prove it matches attention over the whole sequence.Next in this trackThis KV allocator passes ordinary append tests but corrupts forked requests. Find two bugs and prove the repair.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.