Advanced

Draft Heads: Medusa, EAGLE and Multi-Token Prediction

Medusa, EAGLE and multi-token prediction all draft from the target's hidden states instead of running a separate model. This page prices each design in bytes per draft step, explains why Medusa's acceptance decays with position while EAGLE's holds up, and shows how tree verification lowers the batch size where speculation stops paying.

Get full Premium access · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

RELATED CONCEPTS

Speculative Decoding: The Algebra and the Proof The Roofline Model for LLM Inference The KV Cache: Bytes per Token and the Memory Budget Expert Parallelism and the MoE All-to-All Serving Engine Knobs and Their Defaults

PRACTICE THIS IN REAL QUESTIONS

Speculative & Constrained DecodingDerive the expected speedup of speculative decoding, prove it does not change the output distribution, and tell me when it stops paying.→Attention KernelsWrite the causal mask for a decode step with a KV cache, where the query is shorter than the keys. Which corner does it align to?→Speculative & Constrained DecodingImplement speculative-decoding verification for temperature zero. How do you handle the bonus token and the rejected drafts' KV?→Speculative & Constrained DecodingMedusa heads or a separate draft model for a 70B target? Compare acceptance, cost per step and the training each one needs.→Inference Physics & RooflineA vendor claims 1,000 tokens per second per user for a 70B model on one H200. Sanity-check that from the roofline.→Speculative & Constrained DecodingBuild the tree attention mask that verifies a draft tree in one pass. How many tokens does each step verify, and at what cost?→