Skip to content
← Distributed & Disaggregated Inference / 24
expertNewxAIAnthropicOpenAI

Record MoE routing in the rollout engine so the trainer can replay it. What does it cost per token, and what breaks?

An MoE trainer recomputing routes does not always pick the engine's experts, and recording the engine's choices fixes that at a price. The answer sizes the record per token, finds where it really costs, and lists the engine paths that hand the trainer routes that never produced the tokens.

Updated Nov 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

An MoE trainer recomputing routes does not always pick the engine's experts, and recording the engine's choices fixes that at a price. The answer sizes the record per token, finds where it really costs, and lists the engine paths that hand the trainer routes that never produced the tokens.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackTwo SGLang data-parallel ranks deliver well under twice one rank, and half the requests have a bad TTFT. Diagnose it.Next in this trackA video diffusion model runs out of memory on longer clips. What does sequence parallelism save, and what does it cost?Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.