awq

Weight-only INT4 shrinks the bytes but keeps the arithmetic in BF16. The answer finds the batch size where that bargain flips, which does not depend on the model, then explains why a code benchmark notices before perplexity does.

The two methods ship identical file formats and run on identical kernels, so the difference is entirely in how the rounding was chosen. The answer shows the one design choice that makes one of them fragile when traffic stops looking like the calibration set.