Skip to content
← Speculative & Constrained Decoding / 08
mediumNewAnthropicSnowflakeDatabricks

When does n-gram or prompt-lookup speculation work well with no draft model at all? Walk me through a worked acceptance example.

A drafter that costs nothing and needs no training sounds too good, and on open chat it does nothing. The answer traces a code-rename request step by step, counts accepted tokens under three match-length settings, and says when to turn it off.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

A drafter that costs nothing and needs no training sounds too good, and on open chat it does nothing. The answer traces a code-rename request step by step, counts accepted tokens under three match-length settings, and says when to turn it off.

an account raises the free count per topic
UP NEXT ON YOUR JOURNEY
Next in this trackYou turned on speculative decoding in production. What do you measure to prove it helps, and what is the right baseline to compare against?Next in this trackHow does grammar-constrained decoding build the allowed-token mask at every step, and what does that cost per step at batch 128?Next in this trackServe JSON-schema outputs at high throughput: what do you precompile, what do you cache, and what is the tokenizer boundary problem?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.