hardNewOpenAIAnthropicNVIDIA

Implement cancellation for an inference request without reusing KV pages while GPU work still references them.

A disconnect arrives after launch, then the request ID is reused. Implement deferred reclamation and prove that late or duplicate completions cannot emit stale output or free another request's pages.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

A disconnect arrives after launch, then the request ID is reused. Implement deferred reclamation and prove that late or duplicate completions cannot emit stale output or free another request's pages.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 136 remaining answers · ₹2,000 / $25

Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY

Next in this trackA continuous batch changes order while a forward pass runs. How do you return each token to its owner?Next in this trackA speculative verifier returns correct tokens, but the next step diverges. Debug its cache commit boundary.Next in this trackLogits are identical, but a seeded request changes output after batch reordering. Repair the sampler's RNG contract.

FEDITOR'S NOTE

Stopping a response and reclaiming its buffers are separate transitions. State which completion events prove the last consumer is finished, including transfers outside the model's compute stream.

DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.