A 60% MBU number is a symptom, not a diagnosis. The answer rebuilds the floor from bytes, then splits the missing 40% into buckets a profiler can confirm and a team can own.
an account raises the free count per topic
← Inference Physics & Roofline / 09
hardNewOpenAIAnthropicNVIDIA
Your decode step runs at 60% of the bandwidth floor. Where does the rest of the time go? Build me the budget.
A 60% MBU number is a symptom, not a diagnosis. The answer rebuilds the floor from bytes, then splits the missing 40% into buckets a profiler can confirm and a team can own.
Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.
UP NEXT ON YOUR JOURNEY
Next in this trackLong context: at what sequence length do KV reads overtake weight reads for a 70B GQA model at batch 16?Next in this trackDerive the curve of decode tokens per second against batch size, and show me where the knee is.Next in this trackEstimate the energy per output token for a 70B model on H100s from power and throughput. What moves the number most?
DISCUSSION · 0
Nobody has commented on this answer. Add a correction or another way to work it.