Skip to content
← Attention Kernels / 14
hardNewNVIDIAOpenAIAnthropic

Your attention kernel is exact in FP32 and wrong in BF16 at long context, but fine at short context. Find the accumulation bug.

An error that grows with context length is a running sum losing increments, and BF16 stops adding 1 to a total at 256. The answer turns the symptom into a bisection, finds the state held in the wrong dtype, and shows the error curve that proves it.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

An error that grows with context length is a running sum losing increments, and BF16 stops adding 1 to a total at 256. The answer turns the symptom into a bisection, finds the state held in the wrong dtype, and shows the error curve that proves it.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackDesign attention for a 1M-token context on one 8-GPU node: the kernel, the parallelism across GPUs, and the memory plan.Next in this trackA decode-attention kernel wins at 128K context. What evidence would justify enabling it for other shapes?Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.