← Distributed & Disaggregated Inference / 16

expertNewNVIDIASarvam AITogether AI

Design MoE dispatch and combine kernels across NVLink and RDMA: what do they send, and how many SMs do they get?

Counting the all-to-all bytes is the easy half. The kernel design decides how many of those bytes cross the network, which layout the expert GEMM receives, and how many SMs stop doing math, and the right answer differs between prefill and decode.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY

Next in this trackMLA at TP=8 keeps eight copies of its KV latent. How does decode context parallelism shard it, and what does it send?Next in this trackIn prefill/decode disaggregation, should decode pull the KV or prefill push it? What does each choice cost?Next in this trackShould a vision-language model's image encoder run on its own instances? Size the encoder work and its cache, then decide.

DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.