Skip to content
← CUDA, Triton & GPU Performance / 06
mediumNewNVIDIAOpenAIAnthropic

A GEMM launches 140 output tiles on a 132-SM H100. What does that cost, and how do you fix it?

Eight tiles too many turns a one-wave GEMM into two. The answer prices the waste in microseconds for a real prefill shape, then compares the three fixes, including the one that lives in the scheduler, not the kernel.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Eight tiles too many turns a one-wave GEMM into two. The answer prices the waste in microseconds for a real prefill shape, then compares the three fixes, including the one that lives in the scheduler, not the kernel.

an account raises the free count per topic
UP NEXT ON YOUR JOURNEY
Next in this trackWrite a fused RMSNorm kernel in Triton and count its memory traffic against the unfused PyTorch version.Next in this trackFused MoE kernels sort tokens by expert before the GEMM. Why, and what does a grouped GEMM do that a loop over experts cannot?Next in this trackIn a W4A16 GEMM, where does the INT4 unpacking happen, and how does the kernel keep that work hidden under the memory loads?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.