Skip to content
← Implement-from-Scratch Coding / 28
hardNewOpenAINVIDIATogether AI

Implement top-k and top-p sampling on the GPU for a batch of 256 over a 201,088-token vocabulary, without sorting.

Sorting 51 million logits every decode step to keep the top 25 is the answer that gets you a follow-up. This one draws exactly from the truncated distribution with one-pass reductions, proves it, counts the rounds, and then fixes the 90 MB the default tensor-parallel path moves before sampling starts.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Sorting 51 million logits every decode step to keep the top 25 is the answer that gets you a follow-up. This one draws exactly from the truncated distribution with one-pass reductions, proves it, counts the rounds, and then fixes the 90 MB the default tensor-parallel path moves before sampling starts.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackIn CUDA, count 1M MoE routing pairs per expert and scatter-add expert outputs back into tokens when one expert is hot.Next in this trackImplement a LoRA linear layer by hand, then decide whether to merge the adapter into a BF16 base and into an FP8 base.Next in this trackImplement DeepSeek V3's MoE router from its config, then group 4,096 tokens by expert. What must the tests catch?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.