hardNewNVIDIAOpenAIAnthropic

A continuous batch changes order while a forward pass runs. How do you return each token to its owner?

The tensor has the right shape and every token ID is valid, yet one request receives another request's continuation. Reproduce the routing error and repair the launch-to-completion contract.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

The tensor has the right shape and every token ID is valid, yet one request receives another request's continuation. Reproduce the routing error and repair the launch-to-completion contract.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY

Next in this trackA speculative verifier returns correct tokens, but the next step diverges. Debug its cache commit boundary.Next in this trackLogits are identical, but a seeded request changes output after batch reordering. Repair the sampler's RNG contract.Next in this trackA streaming client stops reading. Keep its output bounded without blocking the inference batch.

FEDITOR'S NOTE

The next review question is whether sampling parameters and position metadata use the same permutation as the token inputs. Correct output routing cannot repair a forward pass that read another request's KV.

DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.