Skip to content
← CUDA, Triton & GPU Performance / 15
expertNewNVIDIAOpenAITogether AI

A kernel runs 20% slower on an H200 than on an H100, even though the H200 has 43% more memory bandwidth. How can that happen?

On paper an H200 SXM is an H100 with more and faster memory, so nothing should get slower. The answer works through the four ways it can anyway, from the spec sheet you did not read to the tuning table that does not know the new device's name.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

On paper an H200 SXM is an H100 with more and faster memory, so nothing should get slower. The answer works through the four ways it can anyway, from the spec sheet you did not read to the tuning table that does not know the new device's name.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?Popular right nowDerive the FLOPs per token for a dense transformer from its config. Where does the 2N rule stop being accurate?Popular right nowWhy does prefill turn compute-bound after a few hundred tokens on an H100, and how long does a 4K-token prompt take on a 70B model?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.