Skip to content

The inference concept map

36 concepts across 12 tracks, drawn with the 132 links between them. 97 of those links (73%) cross tracks, which is the part worth looking at: the concepts that decide inference interviews are rarely the ones that sit neatly inside one topic. Hover a concept to see only what it touches. Click to read it.

Graph of 36 inference concepts in 12 tracks with 132 dependency edges. The same concepts appear as a plain list after the graph.The Roofline Model for…The KV Cache: Bytes pe…Paged KV Memory and Bl…Iteration-Level Schedu…The GPU Memory Hierarc…Capacity Planning in T…Autoscaling LLM Fleets…Latency Decomposition:…
same trackcrosses tracks free membersbigger dot = more connections36 shown

Where the map is dense

The most-connected concepts are the ones the rest of the library keeps reaching for, which makes them the highest-leverage things to be solid on: The Roofline Model for LLM Inference (15), The KV Cache: Bytes per Token and the Memory Budget (15), Prefill Versus Decode: Two Workloads on One GPU (11), Capacity Planning in Tokens per Second (10), and Latency Decomposition: Queue, Prefill, Decode (10). If you are deciding where to spend a week, start with a hub rather than a leaf.

Every concept, by track

Inference Physics3

KV Cache & Memory3

Batching & Scheduling3

Attention Kernels3

GPU Performance3

Quantization & Numerics3

Decoding Algorithms3

Distributed Inference3

Serving Engines3

Inference System Design3

Performance Debugging3

Implementation Patterns3

The map is for orientation. If you would rather be told what to do in order, the start-here path sequences the same material by background and stage, and the study paths connect concepts to experiments and worked questions.