Skip to content
← Attention Kernels / 11
mediumNewMistral AIOpenAIGoogle DeepMind

Sliding-window attention in a tiled kernel: which blocks get skipped, and what speedup follows at 32K context with a 4K window?

A 4K window over 32K tokens keeps one eighth of the keys, and the speedup over causal attention is not 8x. The answer counts tiles exactly, classifies which need masking, and explains where the rest of the gap goes.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

A 4K window over 32K tokens keeps one eighth of the keys, and the speedup over causal attention is not 8x. The answer counts tiles exactly, classifies which need masking, and explains where the rest of the gap goes.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackShould RoPE run inside the attention kernel or as a separate pass? Count the bytes saved, then find the precision trap at long positions.Next in this trackVariable-length batches without padding: what is cu_seqlens, how does the kernel index it, and what is the bug that mixes sequences?Next in this trackYour attention kernel is exact in FP32 and wrong in BF16 at long context, but fine at short context. Find the accumulation bug.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.