capacity

The formula takes one line. The answer interviewers score is the memory budget around it: what the engine reserves first, what tensor parallelism does to the heads, and why the count you get is a ceiling.

Doubling the GPUs past the KV head count halves the weights per GPU and leaves the KV per GPU exactly where it was. The answer derives that, prices the duplicate copy, and picks the parallelism that does not pay it.

Half the layers capped at a 128-token window sounds like a big saving at 128K. The arithmetic says the saving is set by the layer ratio, not the window, and the allocator decides whether you get it at all.

Count the blocks with and without sharing and the saving looks large. The answer that scores also knows when the copy fires, what a misaligned prompt does, and the workload where the saving all but disappears.

All three policies keep the server alive. They differ in who waits, who redoes work, and which percentile absorbs the damage, and the numbers for a 6K-token victim make the choice for you.

Prompts fit, but their outputs exhaust the pool. Work through a page-level memory ledger, choose the next diagnostic check, and test a concurrency fix against queueing and goodput.