triton

A matrix-vector product has one job: keep enough bytes in flight. The answer counts how many that is per SM, then finds the three places a decode GEMV quietly drops below it.

RMSNorm does almost no arithmetic, so its cost is bytes, and the eager PyTorch version moves nine times more of them than it needs to. The answer writes the one-pass kernel, tests it on awkward hidden sizes, and says where fusion stops paying.

A kernel time is a distribution measured under conditions, and a reviewer will ask about both. The answer builds the timing loop, says what each line defends against, and adds the one sanity check that catches most bad numbers.

The scheduler did not change; the code that turns its decisions into GPU tensors did. The answer covers where per-request state now lives, why Triton became a hard dependency, and the feature list that silently puts you back on the old runner.

The tutorial kernel loads a whole row into one block and breaks the moment the row is a 128,256-entry vocabulary. The answer writes the looped version, keeps it free of NaNs when a whole tile is masked, and tests it the way a grader will.