Skip to content
← Distributed & Disaggregated Inference / 17
hardNewNVIDIAFireworks AITogether AI

MLA at TP=8 keeps eight copies of its KV latent. How does decode context parallelism shard it, and what does it send?

TP gives MLA's shared latent no relief, and DP attention gives it relief at the cost of replicated weights and lockstepped ranks. The answer shards the cache by token across the same TP ranks, prices the three collectives each layer adds, and finds the edge case that breaks a naive merge.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

TP gives MLA's shared latent no relief, and DP attention gives it relief at the cost of replicated weights and lockstepped ranks. The answer shards the cache by token across the same TP ranks, prices the three collectives each layer adds, and finds the edge case that breaks a naive merge.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackIn prefill/decode disaggregation, should decode pull the KV or prefill push it? What does each choice cost?Next in this trackShould a vision-language model's image encoder run on its own instances? Size the encoder work and its cache, then decide.Next in this trackAttention-FFN disaggregation for MoE decode: why ping-pong micro-batches, how many, and when does it beat wide EP?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.