Skip to content
Practice tests · 30 items

Performance Debugging & Benchmarking practice test

p99 TTFT regressions with flat utilisation, OOMs under continuous batching, numerical divergence after an engine upgrade, nondeterministic outputs, slow replicas, and benchmark methodology that does not lie about warm-up, clocks or prompt mix. The pool for this topic holds 10 easy, 12 medium and 8 hard items. Each answer comes with its reasoning and a link to the question or concept page it was drawn from.

Configure a test
Topic
Difficulty
Questions
10 in this pool · about 7 min
Reveal answers
Sign in to startNeeds a free account · retakes deal unseen items first

Sample items with answers

easy · sample
TTFT doubled right after a prompt-template deploy. Request rate, prompt lengths and GPU utilization look unchanged. What is the first suspect?
The new template uses more output tokens, so each request now spends longer in the decode phase
The GPUs started thermal throttling at the same time, which lowered the clocks for every prefill
The tokenizer was upgraded with the template, and the new one is slower on the CPU for every request
✓A per-request value such as a timestamp now sits ahead of the shared prompt, breaking prefix hits

An early per-request token can invalidate that prefix block and every chained descendant. In the synthetic case with 2,560 shared and 440 unique tokens, a 95% shared-token reuse rate gives 440 + 0.05 x 2,560 = 568 computed tokens/request; zero reuse gives 3,000, a 5.282x work ratio. Compare tokenized prompts and cache identity, then matched per-replica counter deltas. The work ratio does not predict a p99 multiplier, and unchanged utilization does not establish unchanged useful work.

easy · sample
Users report choppy streaming: inter-token latency is bad, but time to first token is fine. Where should the investigation focus?
✓The decode step: weight and KV reads, host overhead, prefill chunks packed into it
The prefill path: tokenization, queueing and the prompt's forward pass before token one
The network path: TLS setup and gateway connection handling between client and server
The prefix cache: low hit rates, which make every streamed token slower to compute

Inter-token latency is the time of one engine step, so decompose the step. For Llama 3.1 8B in BF16 at batch 64 and 4K context on an H100 SXM the bandwidth floor is about 15.1 ms; a step that also carries a 2,048-token prefill chunk takes about 79 ms. A bimodal ITL histogram with its p50 near the floor points to prefill interference; a uniformly slow step points to host overhead or a bigger batch. TTFT being fine already clears the prefill path and the prefix cache.

After the test

Picking an option is easier than deriving the answer at a whiteboard. These pages make you derive it: