TL;DR: vLLM keys each 16-token block by a chained hash in a flat map; SGLang stores token sequences on the edges of a radix tree and evicts leaves. The granularity gap is small (a divergence at token 300 hits 288 tokens in vLLM and 300 in a page-1 tree), so the real differences are elsewhere: SGLang offers five eviction policies and cache-aware queue ordering, while vLLM's content-addressed hashes are identical across processes, which is what cross-node KV stores and KV-aware routers key on.
How to approach it
Pin versions first: vLLM v0.31.0 and SGLang v0.5.21 (October 2026), because SGLang's tree core moved to Rust in that release and both projects changed defaults this year. Then compare along three axes in order: what the key is, how a lookup proceeds, and what gets evicted. Finish with workloads, and ask which one the interviewer cares about: chat, agents, or parallel sampling and search.
A strong answer
Data structures. In vLLM a block's identity is sha256(parent_hash, 16 token ids, extra keys), stored in a map from hash to physical block. Lookup walks the request's precomputed hashes and stops at the first miss, one dictionary probe per block. In SGLang the cache is a radix tree whose edges hold token runs and whose nodes hold the KV indices for those tokens. Lookup walks from the root comparing tokens, splitting a node when a request diverges in the middle of an edge. Matching is rounded down to page_size, which the server resolves per attention backend and which can be one token.
Granularity, in numbers. vLLM only hashes full blocks, so a divergence costs the partial block:
shared prefix vLLM hit (16-token blocks) tree hit (page 1)
40 tokens 32 (80%) 40
300 tokens 288 (96%) 300
1,000 tokens 992 (99.2%) 1,000
mean loss per divergence = (0 + 1 + ... + 15) / 16 = 7.5 tokens
On long prompts this is noise. It matters only when shared prefixes are short, as in many few-shot variants that diverge after a few dozen tokens.
Eviction. vLLM frees a finished request's blocks in reverse order to the tail of one free queue, and allocation pops the head, so eviction is LRU over freed blocks and a sequence's tail goes before its head. Blocks still referenced are never in the queue. SGLang evicts from a heap of evictable leaves (lock count zero, no children) and walks upward as parents become leaves, so a shared system prompt with live children can't be evicted at all. The ordering is a policy: --radix-eviction-policy defaults to lru and accepts lfu, slru, priority and tlru (a tail-latency-aware LRU from arXiv 2510.15152). vLLM v0.31 has one built-in order.
Scheduling. SGLang can reorder its waiting queue by longest prefix match (--schedule-policy lpm) so requests that share a node run together and the node stays hot. The default in v0.5.21 is fcfs, and the cache-aware sort switches itself off above 128 waiting requests because matching every queued request each step costs CPU. vLLM's scheduler is FCFS or priority and does not reorder by cache hit.
Identity outside the process. vLLM's hashes are content addresses. With a cryptographic hash the first block's seed is fixed in v0.31, so two replicas compute the same hash for the same prefix. That is the property external KV stores (LMCache, Mooncake connectors) and KV-aware routers rely on, and vLLM publishes block-stored and block-removed events in those terms. A radix tree node is a local object; to share it across nodes SGLang has to derive hashes too, and its KV-event path computes them per page.
| vLLM v0.31 block hashes | SGLang v0.5.21 radix tree | |
|---|---|---|
| Key | Chained hash per 16-token block | Token runs on tree edges |
| Lookup | One map probe per block | Walk edges, compare tokens, split |
| Match granularity | Full blocks only | page_size |
| Eviction | LRU over freed blocks | Leaf heap; lru default, four other policies |
| Queue ordering | FCFS or priority | fcfs default, lpm opt-in |
| Cross-node identity | Built in | Exported per page |
Where each wins. Pick SGLang's tree for programs that branch heavily and rejoin, such as tree search, many parallel samples followed by more prompting, and agents that fork a shared context, especially when you want LFU or priority eviction to protect a few hot prefixes. Pick vLLM's hashes when reuse has to cross replicas or tiers, such as a fleet behind a KV-aware router, or offloading to CPU or remote storage, because the key is meaningful everywhere without a tree to synchronize.
The reversal: once routing is prefix-aware and prompts are long, both caches hit within a percent of each other on the same traffic, and the choice moves to other engine features. Don't pick an engine on cache structure alone for plain chat.
What interviewers probe next
"Isn't the tree slow in Python?" That was the standard criticism. v0.5.21 runs the tree core in Rust by default, falling back to Python only for configurations it does not support.
"A thousand requests share one system prompt. Can either cache evict it under pressure?" Not while any of them is running: vLLM's reference count keeps the blocks out of the free queue, and SGLang's lock count keeps the node out of the leaf heap. Pressure has to come from preemption, not eviction.
"Parallel sampling with n=8." Both share the prompt KV across the eight children; the tree records each branch as a sibling leaf, while vLLM gives each child its own unhashed tail blocks until they fill.
How it's scored
Weak: says the radix tree finds longer prefixes and stops there, or calls vLLM's cache hash-per-prompt.
Hire: compares chained block hashes with edge-token trees, explains reference counts versus lock counts, and puts a number on the granularity gap.
Strong Hire: shows the gap is small on long prompts, then moves to eviction policies, lpm and its 128-request cutoff, and cross-node identity, with versions.
Common mistakes
- Claiming SGLang orders by longest prefix match by default; in v0.5.21 the default is
fcfs. - Treating eviction as "remove the least recent request", when both systems evict shared parents last.
- Ignoring the
page_sizesetting, then assuming the tree always matches token by token. - Choosing by hit rate on a synthetic benchmark of identical prompts, where both caches hit almost everything.