admission control

The two targets are coupled: the TPOT target caps the prefill chunk, which caps prefill throughput, which caps the backlog the TTFT target can absorb. The answer builds the step model, derives the admission rule and shows what to do with requests that fail it.

Accepting everything during a one-minute spike leaves the API serving dead requests for two minutes after it ends. The answer does the queue arithmetic, picks the rejection rule, and writes the response that keeps clients from making it worse.

A web-service autoscaler watching GPU utilization will scale this fleet too late and in the wrong direction. The answer turns the four-minute load time into a headroom number, then decides what absorbs the bursts no scaler can catch.

Unused logical KV pages can coexist with exhausted device memory. Reconcile expanded image tokens, retained embeddings and phase-specific peaks before changing a decoder token limit.

A request costs anything from ten tokens to a hundred thousand, and you only learn the real cost when it finishes. The answer reserves at admission, refunds at completion, fails fast on requests that can never fit, and tests the bound the bucket is supposed to guarantee.