Start here

Decode reads every weight once per token. The answer turns that sentence into a millisecond floor, then accounts for the three things that keep a real server above it.

The formula takes one line. The answer interviewers score is the memory budget around it: what the engine reserves first, what tensor parallelism does to the heads, and why the count you get is a ceiling.

The gain is not a constant. It is a statistic of your output-length distribution, and the right token budget is a point on the roofline. The answer derives both, then says where the paper number and the real knee disagree.

FlashAttention does the same arithmetic as naive attention and is several times faster. The answer is a three-line recurrence and one IO count, and interviewers want both.

A matrix-vector product has one job: keep enough bytes in flight. The answer counts how many that is per SM, then finds the three places a decode GEMV quietly drops below it.

Weight-only INT4 shrinks the bytes but keeps the arithmetic in BF16. The answer finds the batch size where that bargain flips, which does not depend on the model, then explains why a code benchmark notices before perplexity does.

Three parts, and most candidates finish one. The algebra fits on a line, the proof fits in three, and the part interviewers remember is the batch size at which the whole trick runs out of spare compute.

Tensor parallelism divides the weight bytes but not the all-reduce count, so each doubling buys less than the one before. The answer prices that fixed cost, then finds the workload where TP=8 wins anyway.

Five processes, one scheduler with no prefill phase, and a busy loop that should never wait on Python. The answer maps each stage to the component that owns it in the current source, not the 2023 paper.

Most designs draw the router first. The answer that passes does the arithmetic first: tokens per second in each phase, replica-seconds per second, a utilization ceiling, then dollars, and finds that the prompt, not the reply, is two thirds of the bill.

Inspect tokenized prompts and cache-counter deltas to diagnose a TTFT regression. A runnable prefix-boundary check and a routing follow-up test whether the evidence actually supports the diagnosis.

Forty lines of Python, and graders run property tests against them. The answer gets the three invariants right first, then the one copy-on-write case most implementations get backwards.