Skip to content
← Speculative & Constrained Decoding / 18
expertNewBasetenSnowflakeAmazon

With probabilistic drafts on JSON-schema traffic, speculative output overweights the drafter's favorite enum value. Why?

Every output still parses, so the grammar mask works. The distribution is what moved. The answer traces one verified position through the mask, the draft check and the correction token, finds the step that loses exactness, and says which drafting mode makes it exact again.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Every output still parses, so the grammar mask works. The distribution is what moved. The answer traces one verified position through the mask, the draft check and the correction token, finds the step that loses exactness, and says which drafting mode makes it exact again.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackBlock verification accepts more drafts than token-by-token rejection with the same drafter. Why, and when does it gain nothing?Next in this trackImplement a JSON-schema token mask: walk the vocabulary through the automaton, pack it into int32 words, apply it to logits.Next in this trackA diffusion LLM emits 15 to 20 tokens per forward. Is it 15x faster than autoregressive decode? Derive where it wins.
EDITOR'S NOTE

The screen is whether the candidate treats a grammar-illegal draft as an ordinary rejection. Candidates who call the replacement a bonus token have the bug in their head already.

DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.