Skip to content
← Distributed & Disaggregated Inference / 19
hardNewOpenAIGoogle DeepMindMeta

Should a vision-language model's image encoder run on its own instances? Size the encoder work and its cache, then decide.

Vision encoders look small next to the language model, and that is the trap: one large image costs nearly half the prefill its tokens cause, and it cannot be chunked. The answer prices the stall, the cache entry and the encoder fleet.

Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

Vision encoders look small next to the language model, and that is the trap: one large image costs nearly half the prefill its tokens cause, and it cannot be chunked. The answer prices the stall, the cache entry and the encoder fleet.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackAttention-FFN disaggregation for MoE decode: why ping-pong micro-batches, how many, and when does it beat wide EP?Next in this trackGrow and shrink a running expert-parallel MoE deployment without a restart: what moves, what pauses, what does it cost?Next in this trackServe Llama 3.3 70B in FP8 on a node of four L40S GPUs with no NVLink: TP=4, two TP=2 replicas, or pipeline parallel?
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.