How does a paged attention kernel gather keys through a block table, and what does that cost compared with contiguous KV?
The indirection adds 4 bytes per 64 KiB of KV, so the byte count barely moves. The answer shows where the real cost hides: dependent loads, tiles that span several pages, and copy engines that want one contiguous box.
Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.
The indirection adds 4 bytes per 64 KiB of KV, so the byte count barely moves. The answer shows where the real cost hides: dependent loads, tiles that span several pages, and copy engines that want one contiguous box.
At batch 1 the step reads every weight once to produce one token per sequence, so arithmetic intensity sits near 1 FLOP per byte and the tensor cores idle.