Skip to content
← Distributed & Disaggregated Inference / 20
expertNewNVIDIAFireworks AIBaseten

Attention-FFN disaggregation for MoE decode: why ping-pong micro-batches, how many, and when does it beat wide EP?

Splitting attention and experts onto separate GPU pools sounds like free utilization. The answer derives why the pools must pass at least three micro-batches back and forth, what that does to per-token latency, and the network check that decides whether the split beats wide expert parallelism at all.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Splitting attention and experts onto separate GPU pools sounds like free utilization. The answer derives why the pools must pass at least three micro-batches back and forth, what that does to per-token latency, and the network check that decides whether the split beats wide expert parallelism at all.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackGrow and shrink a running expert-parallel MoE deployment without a restart: what moves, what pauses, what does it cost?Next in this trackServe Llama 3.3 70B in FP8 on a node of four L40S GPUs with no NVLink: TP=4, two TP=2 replicas, or pipeline parallel?Next in this trackThe disaggregated deployment benchmarks 80% faster than the aggregated one. Prove it is real before anyone rolls it out.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.