hardNewGoogle TPUGoogle DeepMindAnthropic

Serve a 70B on TPU7x Ironwood: give the roofline, explain ragged paged attention, and justify vLLM's small token budget.

On paper Ironwood is a B200-class chip, so the roofline math carries over. What does not carry over is the execution model: static shapes, compiled buckets and one Pallas kernel for every mix of prefill and decode. The answer derives the floor, then explains why the token budget shrinks.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY

Next in this trackA distillation scorer runs out of memory on known student continuations. Do you need another decode engine?Next in this trackA 0.6B policy's rollout step takes 22 s for 256 answers on a 273 GB/s device. What bounds it, and what shortens it?Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?

DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.