05Why do tensor core GEMMs use tiles like 128 by 128 by 64, and what limits the tile size on Hopper?▼hardNewNVIDIAAMDOpenAIunlockedEvery dimension of a GEMM tile is pinned by a different piece of hardware. The answer derives the reuse that sets the M and N sides, the swizzle row that sets K, and the register and shared-memory budgets that stop the tile growing.Open full answer →
06A GEMM launches 140 output tiles on a 132-SM H100. What does that cost, and how do you fix it?▼mediumNewNVIDIAOpenAIAnthropic○ sign inEight tiles too many turns a one-wave GEMM into two. The answer prices the waste in microseconds for a real prefill shape, then compares the three fixes, including the one that lives in the scheduler, not the kernel.Open full answer →
12Blackwell added tensor memory and 2-CTA MMA. What problem does each one solve, and what changes in a GEMM kernel written for it?▼expertNewNVIDIAOpenAIAnthropic◆ premiumHopper's biggest GEMM tiles spend half the register file on the accumulator and feed every operand byte through one SM's shared memory. Blackwell attacks both limits, and the answer counts the bytes each change gives back.Open full answer →
09In a block-scaled FP8 GEMM like DeepSeek V3's, where are the scales applied, and why must accumulation be promoted to FP32?▼expertNewNVIDIAAMDOpenAI○ sign inFine-grained scales let E4M3 carry every tensor, but they change along the reduction axis, so the tensor cores cannot apply them. The answer walks the main loop where they do get applied, prices that work against the MMA, and explains what Blackwell moves into hardware.Open full answer →