Skip to content
← Implement-from-Scratch Coding / 32
hardNewNVIDIATogether AIFireworks AI

Implement top-k sparse decode attention over a paged cache and prove it equals dense attention under the same mask.

Selecting 2,048 of 128,000 cached tokens per query is the easy part. The answer implements selection, the gather through a block table and an online softmax over the gathered set, proves equality with a masked dense oracle, and tests the edge cases where sparse kernels actually break.

Updated Nov 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Selecting 2,048 of 128,000 cached tokens per query is the easy part. The answer implements selection, the gather through a block table and an online softmax over the gathered set, proves equality with a masked dense oracle, and tests the edge cases where sparse kernels actually break.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackImplement one decode step of a Mamba-style layer, then make prefix reuse and speculative rollback work with checkpoints.Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?Popular right nowDecode attention intensity is fixed at the query-to-KV head ratio. Derive that, then tell me what MLA does to it.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.