Skip to content
← Implement-from-Scratch Coding / 33
hardNewNVIDIAAMDTogether AI

Implement one decode step of a Mamba-style layer, then make prefix reuse and speculative rollback work with checkpoints.

An attention cache can be cut at any token; a recurrent state cannot be un-stepped. The answer writes the step with both of its states, then shows why prefix reuse needs snapshots at block boundaries and a rejected draft needs the pre-draft state, with tests against a run from scratch.

Updated Nov 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

An attention cache can be cut at any token; a recurrent state cannot be un-stepped. The answer writes the step with both of its states, then shows why prefix reuse needs snapshots at block boundaries and a rejected draft needs the pre-draft state, with tests against a run from scratch.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?Popular right nowDecode attention intensity is fixed at the query-to-KV head ratio. Derive that, then tell me what MLA does to it.Popular right nowAccount for every gigabyte on a GPU running vLLM: weights, KV pool, activations, CUDA graphs and NCCL buffers.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.