Skip to content
← Performance Debugging & Benchmarking / 22
hardNewNVIDIATogether AIFireworks AI

A trace of a TP=8 decode step shows all-reduce kernels taking 30% of the step. Is NVLink the bottleneck?

The communication row of the trace is a third of the step. The answer prices the bytes an all-reduce actually moves at decode, splits each kernel into waiting and transfer by aligning the ranks, and finds the fix on a GPU that is not on the critical link at all.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

The communication row of the trace is a third of the step. The answer prices the bytes an all-reduce actually moves at decode, splits each kernel into waiting and transfer by aligning the ranks, and finds the fix on a GPU that is not on the critical link at all.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackPrefix-cache hit rate drops from 70% to 40% at peak. Is the cache too small, or is the router scattering prefixes?Next in this trackA rollout replay reports a 1.25 probability ratio before any weight update. What would you check first?Next in this trackA load test says TTFT got faster when we doubled concurrency. What is wrong with the benchmark?
EDITOR'S NOTE

Listen for whether the candidate prices one all-reduce in bytes before blaming the link, and whether they know a collective's kernel duration includes waiting for the last rank to arrive.

DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.