18A continuous batch changes order while a forward pass runs. How do you return each token to its owner?▼hardNewNVIDIAOpenAIAnthropic◆ premiumThe tensor has the right shape and every token ID is valid, yet one request receives another request's continuation. Reproduce the routing error and repair the launch-to-completion contract.Open full answer →
19A speculative verifier returns correct tokens, but the next step diverges. Debug its cache commit boundary.▼hardNewOpenAIAnthropicNVIDIA◆ premiumAccepted, emitted and computed tokens are different counts. Trace rejection, full acceptance, EOS and an output cap to find the state that the next forward pass is allowed to read.Open full answer →