Skip to content
← Distributed & Disaggregated Inference / 21
expertNewFireworks AITogether AIBaseten

Grow and shrink a running expert-parallel MoE deployment without a restart: what moves, what pauses, what does it cost?

Adding GPUs to a wide-EP deployment touches the expert map, the communicators, the all-to-all buffers and the CUDA graphs. The answer walks the two-phase protocol an engine actually runs, and finds the part of the bill that is paid at launch, before any scaling happens.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Adding GPUs to a wide-EP deployment touches the expert map, the communicators, the all-to-all buffers and the CUDA graphs. The answer walks the two-phase protocol an engine actually runs, and finds the part of the bill that is paid at launch, before any scaling happens.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackServe Llama 3.3 70B in FP8 on a node of four L40S GPUs with no NVLink: TP=4, two TP=2 replicas, or pipeline parallel?Next in this trackThe disaggregated deployment benchmarks 80% faster than the aggregated one. Prove it is real before anyone rolls it out.Next in this trackRecord MoE routing in the rollout engine so the trainer can replay it. What does it cost per token, and what breaks?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.