mediumNewGoogle DeepMindMetaNVIDIA

Estimate the energy per output token for a 70B model on H100s from power and throughput. What moves the number most?

Joules per token is power over throughput, and the power term is the one candidates obsess over. The answer shows why the denominator moves the result by about 50x while the numerator is pinned under a datasheet cap.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY

Next in this trackAn 8B model at batch 1 on an H100 should take 4.8 ms per step but takes 12 ms. Name the three likeliest causes, with numbers.Next in this trackRank the decode-time arithmetic intensity of the QKV projection, attention, the MLP and the LM head. Which ones can fusion help?Next in this trackDesign a roofline-based performance model that predicts TTFT and TPOT for any model, GPU and batch. Where will it be wrong?

DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.