paged attention

The bound is a one-line argument about where a sequence can be partly full. The block size is a three-way trade between that waste, kernel efficiency and how finely the prefix cache can match.

Count the blocks with and without sharing and the saving looks large. The answer that scores also knows when the copy fires, what a misaligned prompt does, and the workload where the saving all but disappears.

The indirection adds 4 bytes per 64 KiB of KV, so the byte count barely moves. The answer shows where the real cost hides: dependent loads, tiles that span several pages, and copy engines that want one contiguous box.

Four beams over a 1,000-token prompt cost 600 MiB if each keeps its own cache and as little as 156 MiB if they share blocks. The answer implements the fork, the copy-on-write and the prune, then shows what vLLM actually does today.

Forty lines of Python, and graders run property tests against them. The answer gets the three invariants right first, then the one copy-on-write case most implementations get backwards.

Beam search reshuffles which hypotheses survive every step, and a naive cache copies every survivor's history. The answer forks block tables instead, gets the release order right so the last sharer writes in place, and proves it against a beam search that never pages.

A runnable allocator hides an exception-safety defect and a payload-aliasing defect. Reduce each failure to a short operation sequence, then test ownership and content against separate oracles.