Skip to content
← Implement-from-Scratch Coding / 31
hardNewNVIDIAxAITogether AI

Implement DeepSeek V3's MoE router from its config, then group 4,096 tokens by expert. What must the tests catch?

A few lines of router decide which of 256 experts each token reaches, and the config holds settings that are easy to get subtly wrong. The answer implements the gate from the checkpoint's own rules, tests it against the reference, and shows two bugs that pass a casual test and change every token.

Updated Nov 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

A few lines of router decide which of 256 experts each token reaches, and the config holds settings that are easy to get subtly wrong. The answer implements the gate from the checkpoint's own rules, tests it against the reference, and shows two bugs that pass a casual test and change every token.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackImplement top-k sparse decode attention over a paged cache and prove it equals dense attention under the same mask.Next in this trackImplement one decode step of a Mamba-style layer, then make prefix reuse and speculative rollback work with checkpoints.Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.