Skip to content
← Implement-from-Scratch Coding / 09
mediumNewNVIDIAOpenAIAMD

Write a row-wise softmax kernel in Triton that stays correct when a row is longer than the block size, and test it.

The tutorial kernel loads a whole row into one block and breaks the moment the row is a 128,256-entry vocabulary. The answer writes the looped version, keeps it free of NaNs when a whole tile is masked, and tests it the way a grader will.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

The tutorial kernel loads a whole row into one block and breaks the moment the row is a 128,256-entry vocabulary. The answer writes the looped version, keeps it free of NaNs when a whole tile is masked, and tests it the way a grader will.

an account raises the free count per topic
UP NEXT ON YOUR JOURNEY
Next in this trackImplement a ring all-reduce as a simulation over p ranks, verify it against a direct sum, and count the bytes each rank sends.Next in this trackImplement beam search over a paged KV cache, forking beams by sharing blocks and copying only the tail on write.Next in this trackImplement a per-tenant rate limiter for an LLM API that limits tokens, not requests, using a token bucket.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.