← Performance Debugging & Benchmarking / 18

hardNewNVIDIAFireworks AITogether AI

Prefill/decode disaggregation increased latency. Is KV transfer, admission, or token delivery on the critical path?

Reconstruct token arrival from a request-correlated trace. Separate transfer overlap from decode admission, and show how first-token delivery changes the metric that exposes a handoff stall.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Reconstruct token arrival from a request-correlated trace. Separate transfer overlap from decode admission, and show how first-token delivery changes the metric that exposes a handoff stall.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY

Next in this trackImage requests OOM while the KV pool is mostly free. Separate encoder peaks from decoder memory and choose a fix.Next in this trackA router raises prefix-cache hits but worsens tail latency. Replay its decisions and find the missing state.Next in this trackA streaming benchmark reports fast tokens, but the client pauses. Which timing claims can its trace support?

FEDITOR'S NOTE

Name the event behind each latency metric. A prefill worker's first sampled token, a proxy's first stream chunk and the client's first visible token need not share a timestamp.

DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.