async scheduling

Scheduling step n+1 before step n's tokens exist sounds impossible. The answer shows the placeholder trick that makes it work, what it costs at an end-of-sequence token, and the configurations where vLLM switches it off for you.

Five processes, one scheduler with no prefill phase, and a busy loop that should never wait on Python. The answer maps each stage to the component that owns it in the current source, not the 2023 paper.

The mask is a 16 KB bitmap per request. Whether it costs nothing or stalls every step depends on which thread computes it and what it is allowed to overlap with.

The scheduler did not change; the code that turns its decisions into GPU tensors did. The answer covers where per-request state now lives, why Triton became a hard dependency, and the feature list that silently puts you back on the old runner.

Small models make the GPU step so short that host work stops hiding behind it. The answer shows how to prove that with a timeline instead of a hunch, how much throughput a few milliseconds of Python cost, and which fixes help in which order.