← ⚙️ Serving Engines
Advanced
vLLM, SGLang and TensorRT-LLM Schedulers Compared
All three engines batch at the iteration level, yet they disagree on when a request may hold KV, whether prefill and decode share a step, and what happens when the pool runs dry. This page compares the three schedulers from tagged source, then puts one fixed KV pool and one long prompt through each rule to show where the numbers diverge.
Get full Premium access · $25
Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.
6 months · One payment · No auto-renewal
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
Batching, Scheduling & AdmissionQuantify what continuous batching buys over static batching, then pick the chunked-prefill token budget and defend it.→Serving Engine InternalsWalk one request through vLLM V1, from the HTTP call to the streamed token. Which process owns each step, and why is it split that way?→Performance Debugging & BenchmarkingAfter a prompt-template deploy, p99 TTFT went from about 100 ms to over a second while GPU utilization did not move. Debug it.→Speculative & Constrained DecodingWhat do temperature, top-k and top-p each do to the next-token distribution, and in what order does a serving engine apply them?→Serving Engine InternalsWhat do gpu_memory_utilization, max_model_len and max_num_seqs each control in vLLM, and how do they interact at startup?→Performance Debugging & BenchmarkingTTFT is fine but inter-token latency is bad. Decompose the decode step and rank the causes.→