← ⌨️ Implementation Patterns
Advanced
Sampling: Temperature, Top-k and Top-p
How LLM token sampling works and how to implement it correctly: temperature rescales logits, top-k keeps the k largest, top-p keeps the smallest prefix of the sorted distribution reaching probability p. Covers order of operations, ties at the cutoff, temperature 0, fully masked vocabularies and the cost at full vocabulary size.
Get full Premium access · ₹2,000 / $25
Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.
6 months · One payment · No auto-renewal
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
Speculative & Constrained DecodingDerive the expected speedup of speculative decoding, prove it does not change the output distribution, and tell me when it stops paying.→Implement-from-Scratch CodingImplement a paged KV-cache block manager with copy-on-write forking, then a scheduler step that preempts when blocks run out.→Attention KernelsWrite the causal mask for a decode step with a KV cache, where the query is shorter than the keys. Which corner does it align to?→Speculative & Constrained DecodingWhat do temperature, top-k and top-p each do to the next-token distribution, and in what order does a serving engine apply them?→Implement-from-Scratch CodingImplement grouped-query attention over a preallocated KV cache, with no torch.cat, for both prefill chunks and decode.→Speculative & Constrained DecodingImplement speculative-decoding verification for temperature zero. How do you handle the bonus token and the rejected drafts' KV?→