Skip to content
← Inference Physics & Roofline / 14
expertNewNVIDIAOpenAIAnthropic

Rank the decode-time arithmetic intensity of the QKV projection, attention, the MLP and the LM head. Which ones can fusion help?

Every weight GEMM ties, attention sits at a constant the batch cannot move, and the ranking flips at batch 8. The answer then separates what fusion can fix from what only the model architecture or the batch can.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Every weight GEMM ties, attention sits at a constant the batch cannot move, and the ranking flips at batch 8. The answer then separates what fusion can fix from what only the model architecture or the batch can.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackDesign a roofline-based performance model that predicts TTFT and TPOT for any model, GPU and batch. Where will it be wrong?Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?Popular right nowDerive the FLOPs per token for a dense transformer from its config. Where does the 2N rule stop being accurate?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.