Skip to content
← All PDF guides
FIELD GUIDE 03 / OCTOBER 2026

The Kubernetes Inference Runtime

Route the request. Transfer the state. Keep decoding.

Follow cache-aware routing, prefill and decode pools, multi-node workers and rollouts through a Kubernetes serving graph, with worked capacity and failure cases.

106 pages25 reference chaptersSearchable PDF
Sign in to check download access

The complete PDF requires active paid Premium or an eligible full-access grant. Referral access does not include downloads.

Cover of The Kubernetes Inference Runtime
THE DECISIONS INSIDE

Defend where request state lives, how a failed worker is replaced and what evidence justifies splitting prefill from decode.

  1. 01Map request, cache and control traffic
  2. 02Budget KV transfer and pool ratios
  3. 03Place multi-node and expert-parallel workers
  4. 04Compare versioned Dynamo and llm-d paths
  5. 05Drain, recover and test the serving graph

Mechanisms and deployment examples retain their source edition and pinned versions. Adapted October 25, 2026.

Opening workbook page from The Kubernetes Inference Runtime. The adjacent topics describe the full edition.
Preview: the opening inference workbook page.
PUT IT TO WORK

Continue on the serving path.

ANOTHER VIEW OF THE SAME SYSTEM

Keep the next reference nearby.

Open Weights, Serving Decisions →

Accelerators for Token Serving →