← 🩺 Performance Debugging
Advanced
Inference Benchmarks That Do Not Lie
Analyze scheduled-arrival and send timestamps, account for failures, and reproduce goodput from a small request trace. Then use an executable queue model to see what a closed-loop client hides before designing a matched-load benchmark.
Get full Premium access · ₹2,000 / $25
Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.
6 months · One payment · No auto-renewal
RELATED CONCEPTS
LESSONS THAT TEACH THIS
Serving under pressure: from admission to releaseTurn a latency target into a request contract10 min→Serving under pressure: from admission to releaseQualify completed requests before accepting a speedup12 min→Serving under pressure: from admission to releaseCapstone: diagnose the worker before releasing it35 min→
PRACTICE THIS IN REAL QUESTIONS
Batching, Scheduling & AdmissionQuantify what continuous batching buys over static batching, then pick the chunked-prefill token budget and defend it.→Batching, Scheduling & AdmissionWhat is goodput for an LLM server, and how can pushing throughput higher make it worse?→Performance Debugging & BenchmarkingTTFT is fine but inter-token latency is bad. Decompose the decode step and rank the causes.→Inference Physics & RooflineA vendor claims 1,000 tokens per second per user for a 70B model on one H200. Sanity-check that from the roofline.→Inference Physics & RooflineWe are moving a 70B model from H100s to B200s. How do decode latency and throughput change, regime by regime?→Batching, Scheduling & AdmissionWhy does p99 TTFT shoot up once utilization passes about 70%? Model it as a queue, then say where that model is wrong for a batching engine.→