Skip to content
← Implement-from-Scratch Coding / 15
expertNewNVIDIAOpenAIAnthropic

Implement split-KV decoding attention and the merge of the partial results, and prove it matches attention over the whole sequence.

At batch 1 a decode attention kernel gives each KV head one thread block and leaves most of the GPU idle. The answer splits the sequence, carries a log-sum-exp with every partial output, merges them exactly, and survives the split that has no keys at all.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

At batch 1 a decode attention kernel gives each KV head one thread block and leaves most of the GPU idle. The answer splits the sequence, carries a log-sum-exp with every partial output, merges them exactly, and survives the split that has no keys at all.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackThis KV allocator passes ordinary append tests but corrupts forked requests. Find two bugs and prove the repair.Next in this trackImplement cancellation for an inference request without reusing KV pages while GPU work still references them.Next in this trackA continuous batch changes order while a forward pass runs. How do you return each token to its owner?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.