Choose a prefill chunk from the next-token deadline
A prefill chunk consumes time that active decodes must tolerate. Derive its maximum from the token-gap budget, then account for the extra decode phases and overhead caused by smaller chunks.
12 MIN · PREMIUM
By InferenceInterviews · Updated
Get full Premium access · ₹2,000 / $25
Every worked answer and derivation in all 12 topics, every concept page, and the full practice-test bank.
6 months · One payment · No auto-renewal