11The same prompt gives different logits at batch 1 and batch 64 on an FP8 server. Why, and how can the kernels prevent it?▼hardNewOpenAIAnthropicGoogle DeepMind◆ premiumReduction order explains the last-bit drift, but a quantized server has a second, much larger leak: a scale computed over the whole batch lets one user's activations change another user's logits. The answer separates the two and fixes both.Open full answer →
06Two identical requests at temperature zero return different outputs. Explain why, and decide what you will promise customers.▼mediumNewOpenAIAnthropicTogether AI○ sign inGreedy decoding is deterministic on paper and the customer has two different answers in hand. The answer finds where the bits change, shows a measured example of it, and turns the finding into a promise the platform can keep.Open full answer →
20Logits are identical, but a seeded request changes output after batch reordering. Repair the sampler's RNG contract.▼hardNewOpenAINVIDIATogether AI◆ premiumA fixed global seed reproduces a launch order, not a request. Build a random-draw address that survives compaction, extra draft work and continuation, then state what it cannot guarantee.Open full answer →