01Your decode GEMV kernel reaches only about half of HBM bandwidth on an H100. Find the reasons and get it above 80%.▼hard★ EssentialNewNVIDIAOpenAIAnthropicunlockedA matrix-vector product has one job: keep enough bytes in flight. The answer counts how many that is per SM, then finds the three places a decode GEMV quietly drops below it.Open full answer →
02In a decode GEMV over a row-major weight matrix, which thread-to-element mapping coalesces, and why does the transposed one not?▼easyNewNVIDIAOpenAIMetaunlockedTwo GEMV kernels do the same arithmetic on the same matrix and differ only in which thread reads which element. One streams at HBM speed; the other turns every warp load into 32 separate transactions.Open full answer →