streaming

Decode each token on its own and an emoji streams as two replacement characters. The answer builds the offset-window decoder that fixes it, runs it, and adds the stop-string holdback that most streaming bugs come from.

About a hundred lines of Python that admit, chunk, preempt, sample and stream. The interviewer is watching where the KV accounting lives and what happens when the pool runs dry.

Four boxes are easy to draw. The answer that passes sizes them first, then says exactly which clock starts where, because a TTFT measured inside the engine and a TTFT measured by the client can disagree by a factor of five.

Voice moves the latency target from the first token to the first audible clause, and the largest item in the budget is not a model at all. The answer allocates 300 ms line by line, then sizes the LLM tier from it.

An event can contain metadata, one token or several tokens. Audit a buffered stream without inventing token timestamps, and separate client launch delay from the engine's execution time.

The stop string arrives as three tokens, the euro sign arrives as two, and the client has already been sent half of both. The answer holds back exactly the text that could still become a stop, decodes bytes incrementally, and tests the stream against a reference that sees the whole output at once.

One slow connection can stall unrelated generations or retain an unbounded output queue. Implement a byte budget that includes the pending write, then connect overflow and disconnects to engine cancellation.