hardNewOpenAIAnthropicFireworks AI

Write the main loop of a minimal LLM serving engine: request queue, scheduler, model step, sampling and streaming.

About a hundred lines of Python that admit, chunk, preempt, sample and stream. The interviewer is watching where the KV accounting lives and what happens when the pool runs dry.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

About a hundred lines of Python that admit, chunk, preempt, sample and stream. The interviewer is watching where the KV accounting lives and what happens when the pool runs dry.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY

Next in this trackWhat changed in vLLM's Model Runner V2 compared with the V1 runner, how does the engine decide which one to use, and what falls back?Next in this trackYou are serving a reasoning model whose answers run 30K output tokens. Which engine do you pick, and how do you configure it?Next in this trackMore CUDA graph captures made decode faster but increased preemption. Diagnose the regression and choose a policy.

DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.