p99 TPOT spikes every few seconds in a regular pattern while p50 stays flat. Find the cause.
A regular period is a clue most candidates throw away. The answer turns it into a ranked list of periodic sources, shows why a few slow steps out of hundreds own the p99, and names the one correlation that separates the top two.
Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.
A regular period is a clue most candidates throw away. The answer turns it into a ranked list of periodic sources, shows why a few slow steps out of hundreds own the p99, and names the one correlation that separates the top two.
At batch 1 the step reads every weight once to produce one token per sequence, so arithmetic intensity sits near 1 FLOP per byte and the tensor cores idle.