preemption

All three policies keep the server alive. They differ in who waits, who redoes work, and which percentile absorbs the damage, and the numbers for a 6K-token victim make the choice for you.

Per token, moving KV over PCIe is several times cheaper than recomputing it, yet vLLM recomputes. The answer derives the break-even bandwidth, shows how small copies fall below it, and names what makes recompute nearly free.

A priority flag fixes the queue and leaves the real damage in place: batch sequences that are already running slow every interactive token. The answer puts a number on the KV the batch class may hold.

About a hundred lines of Python that admit, chunk, preempt, sample and stream. The interviewer is watching where the KV accounting lives and what happens when the pool runs dry.

Short prompts, enormous replies, and a KV footprint that grows every step after admission. The answer sizes the batch from the pool, prices a preemption, and shows why per-user speed, not throughput, is the number to defend.

Compare three capture policies using a memory ledger and a matched workload. Recover the missing KV pages, then defend a policy against both capacity and latency constraints.

A chat fleet sized for its peak sits more than half idle over a day, and a batch API can soak it up. The answer measures the idle replica-hours, finds what limits batch throughput (it is not compute), and decides where the two workloads meet.

Prompts fit, but their outputs exhaust the pool. Work through a page-level memory ledger, choose the next diagnostic check, and test a concurrency fix against queueing and goodput.

Forty lines of Python, and graders run property tests against them. The answer gets the three invariants right first, then the one copy-on-write case most implementations get backwards.