Advanced

Behavior-policy provenance in RL rollouts

A rollout record must identify the policy that sampled each token, the exact token history and the sampling distribution. Learn why weight versions, raw log probabilities and decoded text each leave part of that contract unproven.

Get full Premium access · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

RELATED CONCEPTS

The vLLM V1 Engine Architecture Grammar-Constrained Decoding Prefix Caching and Chained Block Hashes Iteration-Level Scheduling (Continuous Batching)Inference Benchmarks That Do Not Lie

LESSONS THAT TEACH THIS

Inference for RL rollouts: preserve the training signalMap generation and scoring before sizing the worker12 min→Inference for RL rollouts: preserve the training signalPreserve the sampled-token record across turns15 min→Inference for RL rollouts: preserve the training signalPublish weights with an explicit cache boundary18 min→

PRACTICE THIS IN REAL QUESTIONS

KV Cache & GPU MemoryDerive the KV cache footprint for MHA, GQA and MLA, then tell me how many 8K-token conversations fit on four H100s.→Speculative & Constrained DecodingDerive the expected speedup of speculative decoding, prove it does not change the output distribution, and tell me when it stops paying.→KV Cache & GPU MemoryWhy does a paged KV cache waste at most one block per sequence, and how would you pick the block size?→Attention KernelsWrite the causal mask for a decode step with a KV cache, where the query is shorter than the keys. Which corner does it align to?→Quantization & NumericsFP8 comes in E4M3 and E5M2. Give the range and precision of each, and say which tensors use which in inference.→Speculative & Constrained DecodingWhat do temperature, top-k and top-p each do to the next-token distribution, and in what order does a serving engine apply them?→