Snowflake Inference Engineer interview questions

Twenty lines of roofline arithmetic get within striking distance of a real server, and the interviewer knows it. What they are scoring is the list of places the model breaks, and how you would catch each one before a capacity plan rests on it.

Recomputing a prefix costs FLOPs proportional to the model, fetching it costs bytes proportional to the cache. One ratio decides it, and the answer shows by how much fetching wins and the three situations where it still loses.

A priority flag fixes the queue and leaves the real damage in place: batch sequences that are already running slow every interactive token. The answer puts a number on the KV the batch class may hold.

With no latency target, every scheduler knob points the other way from chat serving. The answer sizes the fleet from the roofline, finds that prefill is three quarters of the bill, and lists the levers in order of dollars saved.

Packing removes padding but puts different requests' tokens side by side in one tensor, so the kernel's tile boundaries no longer match sequence boundaries. The answer shows the indexing and a bug that a causal test can never catch.

A drafter that costs nothing and needs no training sounds too good, and on open chat it does nothing. The answer traces a code-rename request step by step, counts accepted tokens under three match-length settings, and says when to turn it off.

A per-request schema compile stalls the scheduler, a per-step mask must beat the forward pass, and skipping forced characters can push the model off its own tokenization. The answer handles all three with numbers from one realistic schema.

Requests-per-minute limits let one tenant with 128K-token prompts take a fifth of a replica's memory per request. The answer prices quotas in the resource tokens actually consume, then isolates the three things tenants share: compute, KV memory and the prefix cache.

Five hundred merged fine-tunes need five hundred GPUs before the first request. Served as adapters on a shared base they need a few dozen. The answer sizes the adapters, places them in a three-tier memory hierarchy, and finds the one setting that silently multiplies their memory by eight.

Offline work changes the regime: the batch is as big as memory allows, latency is irrelevant, and output tokens stop being expensive. The answer prices both passes in GPU-hours, finds which one is the bill, and cuts it.

Perplexity matches the BF16 baseline to two decimals and the needle-in-a-haystack score collapses past 16K. The answer explains why the two checks measure different things, isolates weights, KV and config with a small grid, and shows a measured case of a quantization scheme that only breaks at length.

The accept rule is one line. The residual distribution, the bonus token, rows that drafted fewer tokens and a test that can tell a lossless verifier from a subtly biased one are the rest of the grade.

Beam search reshuffles which hypotheses survive every step, and a naive cache copies every survivor's history. The answer forks block tables instead, gets the release order right so the last sharer writes in place, and proves it against a beam search that never pages.

The formula takes one line. The answer interviewers score is the memory budget around it: what the engine reserves first, what tensor parallelism does to the heads, and why the count you get is a ceiling.

The bound is a one-line argument about where a sequence can be partly full. The block size is a three-way trade between that waste, kernel efficiency and how finely the prefix cache can match.

Weights and KV are the two numbers everyone knows. The answer that gets the offer accounts for the other few gigabytes, says which ones vLLM measures and which it guesses, and knows where an OOM comes from when the sum looks fine.

FP8 halves the cache and the attention bytes, and on short prompts it looks free. The answer explains why the scale matters less than people think for rounding and more than they think for clipping, and why the failures hide at 128K.

Doubling the GPUs past the KV head count halves the weights per GPU and leaves the KV per GPU exactly where it was. The answer derives that, prices the duplicate copy, and picks the parallelism that does not pay it.

Half the layers capped at a 128-token window sounds like a big saving at 128K. The arithmetic says the saving is set by the layer ratio, not the window, and the allocator decides whether you get it at all.

A prefix cache only hits from the first token forward, so evicting the wrong block can strand gigabytes that stay resident and can never be hit. The answer derives how much, and which eviction order guarantees it never happens.

Absorption is one line of matrix associativity. The part interviewers score is the bill: attention gets more expensive per cached token, and the kernel lands in a roofline regime GQA never visits.