Skip to content
← Implement-from-Scratch Coding / 14
mediumNewOpenAIAnthropicPerplexity

Implement a minimal autoregressive generate loop that streams text and honors stop sequences, including ones split across tokens.

The stop string arrives as three tokens, the euro sign arrives as two, and the client has already been sent half of both. The answer holds back exactly the text that could still become a stop, decodes bytes incrementally, and tests the stream against a reference that sees the whole output at once.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

The stop string arrives as three tokens, the euro sign arrives as two, and the client has already been sent half of both. The answer holds back exactly the text that could still become a stop, decodes bytes incrementally, and tests the stream against a reference that sees the whole output at once.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackImplement split-KV decoding attention and the merge of the partial results, and prove it matches attention over the whole sequence.Next in this trackThis KV allocator passes ordinary append tests but corrupts forked requests. Find two bugs and prove the repair.Next in this trackImplement cancellation for an inference request without reusing KV pages while GPU work still references them.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.