Skip to content
← Attention Kernels / 12
hardNewMetaNVIDIAFireworks AI

Should RoPE run inside the attention kernel or as a separate pass? Count the bytes saved, then find the precision trap at long positions.

Fusing RoPE saves a read and a write of Q and K, which is real but small in prefill. The precision bug is not small: one tensor in the wrong dtype scrambles positional information by a few hundred tokens. The answer sizes both.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Fusing RoPE saves a read and a write of Q and K, which is real but small in prefill. The precision bug is not small: one tensor in the wrong dtype scrambles positional information by a few hundred tokens. The answer sizes both.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackVariable-length batches without padding: what is cu_seqlens, how does the kernel index it, and what is the bug that mixes sequences?Next in this trackYour attention kernel is exact in FP32 and wrong in BF16 at long context, but fine at short context. Find the accumulation bug.Next in this trackDesign attention for a 1M-token context on one 8-GPU node: the kernel, the parallelism across GPUs, and the memory plan.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.