continuous batching

The gain is not a constant. It is a statistic of your output-length distribution, and the right token budget is a point on the roofline. The answer derives both, then says where the paper number and the real knee disagree.

Each draft token is a real token in the step's budget, and verification turns a bandwidth-bound step compute-bound sooner. The answer shows the accounting, the rollback that makes rejection cheap, and the batch size where speculation starts losing.

About a hundred lines of Python that admit, chunk, preempt, sample and stream. The interviewer is watching where the KV accounting lives and what happens when the pool runs dry.

A scheduler is a loop over three queues and a budget. The answer gets the order of those loops right, shows what the budget costs in milliseconds of TPOT, and tests the invariants a grader will check.

The tensor has the right shape and every token ID is valid, yet one request receives another request's continuation. Reproduce the routing error and repair the launch-to-completion contract.

A fixed global seed reproduces a launch order, not a request. Build a random-draw address that survives compaction, extra draft work and continuation, then state what it cannot guarantee.

One slow connection can stall unrelated generations or retain an unbounded output queue. Implement a byte budget that includes the pending write, then connect overflow and disconnects to engine cancellation.