Interview chains, by round

95 calculations across 12 rounds. A chain is the arithmetic an interviewer expects you to produce before you draw a box: inputs pinned to a named part, model or engine version, the substitution written out, and one line that says whether the answer is plausible. Each card prints on its own.

• Inference Physics Interview Chains

  The ridge point · The batch-1 decode floor · Prefill time · Where batching crosses the ridge · When KV reads overtake weight reads · A batched decode step · One output token against one prompt token · MBU from a measured step

• KV Cache Interview Chains

  Bytes per token, GQA · Bytes per token, MLA · The pool left for KV · What block rounding wastes · Sliding-window layers · A shared system prompt · Fetch from host memory or recompute · FP8 KV

• Batching and Scheduling Chains

  Little's law for an LLM server · KV in flight and the naive ceiling · Goodput at a TPOT target · Swap or recompute a preempted request · Head-of-line blocking under FCFS · The p99 TTFT knee · A 100K-token prompt under the budget · Tenant cost in weighted tokens

• Attention Kernel Interview Chains

  What FlashAttention removes from HBM · The online softmax on numbers · The log-sum-exp merge · How many splits for decode · The causal mask for a decode step · What the backward pass saves · GQA loads KV once per group · The block-table cost of paged attention

• GPU Performance Interview Chains

  Bytes in flight to reach bandwidth · A coalesced warp load · Occupancy from three limits · Shared-memory bank conflicts · GEMM tile intensity · Wave quantization · Fused RMSNorm traffic · Launch gaps in a decode step

• Quantization Interview Chains

  Bits per weight, scales included · 70B weight bytes by format · Where INT4 W4A16 loses to FP8 · E4M3 against E5M2 · One outlier sets the scale · MXFP4 against NVFP4 scales · A BF16 sum that stops at 256 · FP16 overflow in a norm

• Decoding Interview Chains

  Tokens per verification · Accepted drafts are not alpha · Draft cost from bytes · The best draft length · The verify-token budget at batch · Temperature, then top-p · A grammar mask per step · Beam search on paged KV

• Distributed Inference Chains

  Two all-reduces per layer · What TP=8 does to a 70B step · MoE all-to-all bytes · Does 405B fit on one node · KV transfer for disaggregation · DP attention capacity for MLA · The pipeline bubble

• Serving Engine Interview Chains

  The pool and the max_model_len check · CUDA graph padding · Prefix-cache hit tokens · Block against radix granularity · The overlap scheduler · No-evict admission · Detokenization cost · Multi-LoRA bytes per step

• Inference System Design Chains

  Prefill demand in replica-seconds · Decode work and the TPOT share · Replicas at a utilization ceiling · The prefix-affinity spill rule · Headroom for a cold start · Quota units for prompt and output tokens · One long request's share of a pool · Price per phase from a GPU-hour

• Performance Debugging Chains

  Lost prefix reuse · The inter-token floor and a mixed step · A startup OOM that the totals hide · A preemption page ledger · What FP8 should have bought · A link that trained at the wrong speed · Benchmark 2x, production 10% · A periodic p99 with a flat median

• Implementation Coding Chains

  What concatenation copies · When the allocator needs a block · Why top-p needs a short circuit · The token budget bounds a step · What a radix hit saves · Position ids in BF16 · INT4 packing · Ring all-reduce traffic

The terms on these cards are defined in the glossary, and the study paths describe how to rehearse a chain until it holds up in a loop.