15A kernel runs 20% slower on an H200 than on an H100, even though the H200 has 43% more memory bandwidth. How can that happen?▼expertNewNVIDIAOpenAITogether AI◆ premiumOn paper an H200 SXM is an H100 with more and faster memory, so nothing should get slower. The answer works through the four ways it can anyway, from the spec sheet you did not read to the tuning table that does not know the new device's name.Open full answer →
21A streaming benchmark reports fast tokens, but the client pauses. Which timing claims can its trace support?▼hardNewNVIDIATogether AIFireworks AI◆ premiumAn event can contain metadata, one token or several tokens. Audit a buffered stream without inventing token timestamps, and separate client launch delay from the engine's execution time.Open full answer →