Skip to content
THE LEARNING PATHS

Build intuition.
Then put it to work.

Follow a sequence through the published library, or rehearse the 25-problem interview review. Work each problem before opening its answer, then use the supporting concepts and experiments to check your reasoning.

Check your starting point
FROM REQUEST TO TOKEN

Inference & serving

Build the memory model, follow the scheduler, and reason through an overloaded engine.

Bring with you
Transformer basics and working Python.
Work toward
Diagnose capacity regressions and verify KV ownership, cancellation, and cache isolation.
Start this path
  1. 01
    Concept · Free

    The Roofline Model for LLM Inference

    Separate arithmetic and memory limits.

  2. 02
    Concept · Free

    The KV Cache: Bytes per Token and the Memory Budget

    Account for the bytes a sequence retains.

  3. 03
    Experiment · Free

    KV memory budget

    Change context and concurrency in the memory model.

  4. 04
    Concept · Free

    Iteration-Level Scheduling (Continuous Batching)

    Trace which requests run in each iteration.

  5. 05
    Worked question · Available

    At peak traffic the engine preempts requests constantly. Diagnose it from the metrics and fix it.

    Use a page budget to diagnose preemption.

  6. 06
    Worked question · Premium

    More CUDA graph captures made decode faster but increased preemption. Diagnose the regression and choose a policy.

    Choose graph captures within a KV and latency budget.

  7. 07
    Worked question · Premium

    This KV allocator passes ordinary append tests but corrupts forked requests. Find two bugs and prove the repair.

    Repair shared-page corruption and test the ownership rules.

  8. 08
    Worked question · Premium

    Implement cancellation for an inference request without reusing KV pages while GPU work still references them.

    Stop generation while retaining pages until their last use completes.

  9. 09
    Worked question · Premium

    Implement a KV block allocator that shares prefix blocks across requests by chained hashes, with LRU eviction of cached free blocks.

    Verify reuse within a cache scope and isolation when identity changes.

  10. 10
    Worked question · Premium

    A hybrid model mixes attention and recurrent layers. Budget its state and design safe prefix reuse and rollback.

    Budget recurrent snapshots and restore a prefix valid for every layer.

  11. 11
    Worked question · Available

    After a prompt-template deploy, p99 TTFT went from about 100 ms to over a second while GPU utilization did not move. Debug it.

    Investigate a cache regression from evidence.

A NOTE ON THE CURRICULUM

These are curated study paths through existing material, with access shown at each step. For dedicated lessons, exercises and progress tracking, explore the published courses.

Explore the courses