goodput

Doubling the batch raised tokens per second by a third and sent goodput to zero. The answer shows why every sequence in a batch shares one step time, and how to find the batch size that maximizes the number users feel.

A cheaper GPU and a faster decoder do not automatically produce a cheaper service. Audit a cost ledger, reject misleading wins, and choose a rollout only after quality and latency gates pass.