← Distributed & Disaggregated Inference / 12

mediumNewNVIDIAOpenAIMeta

A multi-node MoE deployment's TPOT doubled overnight, and one network link is degraded. How do you find it, and how do you design around it?

Collectives run at the speed of their slowest link, so one NIC that trained at a quarter of its rate slows every GPU in the instance by the same amount. The answer shows the arithmetic that makes a 4x slower link a 2x slower step, then bisects the cluster to the culprit.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY

Next in this trackEach decode layer needs the previous all-to-all to finish. How do you overlap communication with compute anyway, and at what cost?Next in this trackYou need to get 500 GB of weights onto 1,000 GPUs as fast as possible. What is the lower bound, and what design gets close to it?Next in this trackDesign the parallelism plan for a trillion-parameter MoE when your fleet is half H100 nodes and half B200 nodes.

DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.