Eight tenants, eight adapters, one base model and one batch. The answer explains why the extra cost scales with distinct adapters rather than requests, and which default serializes them.
an account raises the free count per topic
← Serving Engine Internals / 09
hardNewFireworks AITogether AIDatabricks
How does an engine batch requests for different LoRA adapters in one forward pass, and what do Punica-style shrink and expand kernels do?
Eight tenants, eight adapters, one base model and one batch. The answer explains why the extra cost scales with distinct adapters rather than requests, and which default serializes them.
Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.
UP NEXT ON YOUR JOURNEY
Next in this trackWith JSON-schema decoding on, where in the engine step is the grammar mask computed and applied, and what does it cost?Next in this trackYou upgraded the serving engine with the same flags and throughput fell. Which defaults do you diff first, and how do you prove it?Next in this trackllama.cpp versus vLLM: what is each engine built for, and when does the CPU or Apple Silicon path actually win?
DISCUSSION · 0
Nobody has commented on this answer. Add a correction or another way to work it.