Skip to content
← Serving Engine Internals / 14
expertNewRed HatAnyscaleDatabricks

What changed in vLLM's Model Runner V2 compared with the V1 runner, how does the engine decide which one to use, and what falls back?

The scheduler did not change; the code that turns its decisions into GPU tensors did. The answer covers where per-request state now lives, why Triton became a hard dependency, and the feature list that silently puts you back on the old runner.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

The scheduler did not change; the code that turns its decisions into GPU tensors did. The answer covers where per-request state now lives, why Triton became a hard dependency, and the feature list that silently puts you back on the old runner.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackYou are serving a reasoning model whose answers run 30K output tokens. Which engine do you pick, and how do you configure it?Next in this trackMore CUDA graph captures made decode faster but increased preemption. Diagnose the regression and choose a policy.Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.