Skip to content
← Implement-from-Scratch Coding / 25
hardNewNVIDIAAMDMeta

Write a block reduction and an inclusive prefix scan in CUDA C with warp shuffles, correct for any n.

Both kernels are ten lines of shuffles. The round is decided by the lines around them: lanes past n that return early, a float sum that changes bits with the grid size or with atomics, and a three-kernel scan that moves twice the bytes it needs.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Both kernels are ten lines of shuffles. The round is decided by the lines around them: lanes past n that return early, a float sum that changes bits with the grid size or with atomics, and a three-kernel scan that moves twice the bytes it needs.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackWrite a numerically safe row softmax kernel in CUDA C, with an online max and sum merged by warp shuffles.Next in this trackImplement a priority scheduler for a paged KV pool, with preemption and aging so low-priority requests cannot starve.Next in this trackImplement top-k and top-p sampling on the GPU for a batch of 256 over a 201,088-token vocabulary, without sorting.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.