Every token.
Every tradeoff.
Go into your inference engineering interview with the intuition to derive, debug, and defend your decisions.
Start reading. No account required.
Learn it. Question it.
Make it your own.
A derivation gives you the model. An experiment exposes its limits. A hard question asks you to make the call.
Get beneath the abstraction.
Memory layouts, kernel mechanics, and the decisions a serving engine makes for you.
Explore the libraryChange an input. Follow the cost.
Explore decode, KV memory, speculation, and scheduling with interactive teaching models.
Open an experimentTurn intuition into an answer.
Work the problem, compare your reasoning, and tackle the follow-up that changes a constraint.
Find your next questionYour role.
Your next level of depth.
Start with what your work demands. Follow a deliberate sequence through concepts, experiments, and worked questions.
Explore the learning pathsInference & serving
Build the memory model, follow the scheduler, and reason through an overloaded engine.
11 steps · First concept free · Access shown at each step
- Bring with you
- Transformer basics and working Python.
- Work toward
- Diagnose capacity regressions and verify KV ownership, cancellation, and cache isolation.
- A good fit if
- You can explain attention and want to reason about what an engine does with memory and time once requests arrive.
- Not in this path
- Writing kernels and splitting a model across GPUs; the kernel path and the distributed-inference topic cover those.
- 01Concept · Free
The Roofline Model for LLM Inference
Separate arithmetic and memory limits.
- 02Concept · Free
The KV Cache: Bytes per Token and the Memory Budget
Account for the bytes a sequence retains.
- 03Experiment · Free
KV memory budget
Change context and concurrency in the memory model.
- 04Concept · Free
Iteration-Level Scheduling (Continuous Batching)
Trace which requests run in each iteration.
- View all 11 steps
Write it. Run it. Explain it.
Implement a serving component in Python, run it in your browser and inspect the result. Start with the split-KV merge; compare plans when you want the full edge-case suites and reference solutions.
The prompt is short.
The reasoning goes deep.
The weights fit.
The prompts fit.
So why the preemption?
At peak load, the engine keeps evicting sequences. Work from a concrete memory budget and decide what you would measure or change first.
Investigate the case01 Weights stay resident.
02 Prompt KV fills the pool.
03 Decode keeps adding state.
What did admission forget?
Conceptual memory sketch. Open the case for the synthetic evidence and arithmetic.
One serving path.
Twelve ways to go deeper.
Build the cost model.
Defend the decisions.
Work it through.
Keep it close.
You should be able
to check the reasoning.
Show the arithmetic. Name the hardware and the regime. Keep the source attached. When an assumption changes, the answer should change with it.
Read our accuracy standard