Implement-from-Scratch Coding practice test
The coding round of an inference loop: grouped-query attention with a cache, online-softmax attention, a paged block allocator, a continuous batching scheduler, a lossless speculative verifier, top-p sampling, RoPE, INT4 packing and Triton kernels. The pool for this topic holds 10 easy, 12 medium and 8 hard items. Each answer comes with its reasoning and a link to the question or concept page it was drawn from.
Sample items with answers
New query i sits at absolute position S - T + i, because S - T tokens were already cached before this call. Causality lets it see every key at or before its own position, so j <= S - T + i. Writing the mask as j <= i, the formula for a fresh prefill, is the classic bug: during decode, T = 1, and the single query then sees only key 0. With torch.triu the masked region needs diagonal = S - T + 1, not 1. Test T < S explicitly, since the T = S case hides the offset.
cum - probs is the mass strictly before each token, so a token is kept when the mass ahead of it has not yet reached p. That includes the token whose own probability carries the sum past p, which top-p requires. cum < p drops that crossing token: with probabilities [0.6, 0.3, 0.1] and p = 0.5, it keeps nothing at all. cum <= p has the same defect except on exact equality. probs >= p ignores the cumulative mass. The top token always survives under the right rule, since the mass before it is 0.
After the test
Picking an option is easier than deriving the answer at a whiteboard. These pages make you derive it: