Skip to content
← Implement-from-Scratch Coding / 07
mediumNewMetaMistral AINVIDIA

Implement RoPE for a batched decode step where every row sits at a different position, and verify it against the full-sequence version.

RoPE is a rotation per pair of dimensions. The bugs live in which dimensions pair up, which position each row of a ragged batch is at, what left padding does to position ids, and the precision of the angle at position 100,000.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

RoPE is a rotation per pair of dimensions. The bugs live in which dimensions pair up, which position each row of a ragged batch is at, what left padding does to position ids, and the precision of the angle at position 100,000.

an account raises the free count per topic
UP NEXT ON YOUR JOURNEY
Next in this trackImplement INT4 packing with group scales and a dequantizing matmul, and test it against FP32 with odd K and a partial group.Next in this trackWrite a row-wise softmax kernel in Triton that stays correct when a row is longer than the block size, and test it.Next in this trackImplement a ring all-reduce as a simulation over p ranks, verify it against a direct sum, and count the bytes each rank sends.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.