THE LIBRARY

Good questions start
with deeper understanding.

Keep the mechanism close. Explore the concepts, experiments, and guides behind the serving path.

1 resources to exploreRead. Experiment. Put it to work.

PDF guidesPremium · download rights apply

The inference field guide

Eight pages of worked serving decisions: KV reservations, prefill deadlines, cancellation, speculation, benchmark qualification and kernel investigation.

8 pages · 2026-10-21