Advanced

Model Artifact Compatibility

A checkpoint, an exported graph and a compiled engine carry different contracts. Trace the files, shapes, precision and runtime identity that must agree before a serving replica can safely accept traffic.

Get full Premium access · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

RELATED CONCEPTS

The vLLM V1 Engine Architecture Serving Engine Knobs and Their Defaults Number Formats for Inference: BF16, FP8, INT4, FP4 CUDA Graphs and Launch Overhead in Decode Inference Benchmarks That Do Not Lie

PRACTICE THIS IN REAL QUESTIONS

Serving Engine InternalsWhy do serving engines capture a CUDA graph per batch size, and what does padding to the nearest captured size actually cost?→Serving Engine InternalsTensorRT-LLM in 2026: what is the PyTorch backend, how does its in-flight batching admit requests, and when do you still pick it?→Quantization & NumericsGGUF k-quants and EXL2 mix bit-widths inside one model. What do they trade, and where do they fit next to datacenter GPU formats?→Inference System DesignDesign the rollout of a new model version so that neither quality nor latency regresses in production. How do you know when to promote?→Attention KernelsWhich attention kernel will vLLM and SGLang actually run on an H100 and a B200, and what silently changes the answer?→