Advanced

Property-Testing Engine Components

How to test serving-engine code the way graders do: state invariants, generate random operation sequences, compare against a slow reference within a tolerance derived from the number format, and shrink failures to a minimal repro. Includes a runnable stateful test for a KV block pool.

Get full Premium access · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

RELATED CONCEPTS

Attention With a KV Cache, From Scratch Paged KV Memory and Block Tables Sampling: Temperature, Top-k and Top-p Nondeterminism and Batch Invariance The Online Softmax Recurrence Inference Benchmarks That Do Not Lie

LESSONS THAT TEACH THIS

Serving under pressure: from admission to releaseCancel the request without reusing a live page12 min→Serving under pressure: from admission to releaseCapstone: diagnose the worker before releasing it35 min→

PRACTICE THIS IN REAL QUESTIONS

Attention KernelsDerive online softmax, prove it is exact, and explain why it makes FlashAttention fast without reducing FLOPs.→Speculative & Constrained DecodingDerive the expected speedup of speculative decoding, prove it does not change the output distribution, and tell me when it stops paying.→Implement-from-Scratch CodingImplement a paged KV-cache block manager with copy-on-write forking, then a scheduler step that preempts when blocks run out.→Attention KernelsWrite the causal mask for a decode step with a KV cache, where the query is shorter than the keys. Which corner does it align to?→Implement-from-Scratch CodingImplement grouped-query attention over a preallocated KV cache, with no torch.cat, for both prefill chunks and decode.→Attention KernelsImplement online softmax for a whole tile of queries, not one query, and write the rescaling step in matrix form.→