Skip to content
← Attention Kernels / 07
mediumNewAnyscaleRed HatFireworks AI

How does a paged attention kernel gather keys through a block table, and what does that cost compared with contiguous KV?

The indirection adds 4 bytes per 64 KiB of KV, so the byte count barely moves. The answer shows where the real cost hides: dependent loads, tiles that span several pages, and copy engines that want one contiguous box.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

The indirection adds 4 bytes per 64 KiB of KV, so the byte count barely moves. The answer shows where the real cost hides: dependent loads, tiles that span several pages, and copy engines that want one contiguous box.

an account raises the free count per topic
UP NEXT ON YOUR JOURNEY
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.