Advanced

Diffusion Inference Critical Path

Separate one image's dependent denoising steps from pipeline overlap across requests. Derive latency and throughput, account for classifier-free guidance, and test the quality cost of cached or skipped computation.

Get full Premium access · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

RELATED CONCEPTS

The Roofline Model for LLM Inference Model Artifact Compatibility Number Formats for Inference: BF16, FP8, INT4, FP4 Inference Benchmarks That Do Not Lie Tensor Parallelism for Serving

PRACTICE THIS IN REAL QUESTIONS

Batching, Scheduling & AdmissionQuantify what continuous batching buys over static batching, then pick the chunked-prefill token budget and defend it.→Serving Engine InternalsWalk one request through vLLM V1, from the HTTP call to the streamed token. Which process owns each step, and why is it split that way?→Inference System DesignSize the fleet for a 70B chat API at 200 req/s, 2K tokens in and 300 out, p99 TTFT under 1 s, TPOT under 50 ms. What is the token cost?→Performance Debugging & BenchmarkingAfter a prompt-template deploy, p99 TTFT went from about 100 ms to over a second while GPU utilization did not move. Debug it.→KV Cache & GPU MemoryWhy does a paged KV cache waste at most one block per sequence, and how would you pick the block size?→Performance Debugging & BenchmarkingTTFT is fine but inter-token latency is bad. Decompose the decode step and rank the causes.→