Two identical requests at temperature zero return different outputs. Explain why, and decide what you will promise customers.
Greedy decoding is deterministic on paper and the customer has two different answers in hand. The answer finds where the bits change, shows a measured example of it, and turns the finding into a promise the platform can keep.
Updated Oct 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.
Greedy decoding is deterministic on paper and the customer has two different answers in hand. The answer finds where the bits change, shows a measured example of it, and turns the finding into a promise the platform can keep.
At batch 1 the step reads every weight once to produce one token per sequence, so arithmetic intensity sits near 1 FLOP per byte and the tensor cores idle.