12vLLM turns on async scheduling by default. What does it overlap, what does it need from the model runner, and when must it be off?▼hardNewRed HatNVIDIAAnyscale◆ premiumScheduling step n+1 before step n's tokens exist sounds impossible. The answer shows the placeholder trick that makes it work, what it costs at an end-of-sequence token, and the configurations where vLLM switches it off for you.Open full answer →
14What changed in vLLM's Model Runner V2 compared with the V1 runner, how does the engine decide which one to use, and what falls back?▼expertNewRed HatAnyscaleDatabricks◆ premiumThe scheduler did not change; the code that turns its decisions into GPU tensors did. The answer covers where per-request state now lives, why Triton became a hard dependency, and the feature list that silently puts you back on the old runner.Open full answer →