GPTQ and AWQ both produce INT4 weights with group scales. What does each one optimize, and when does one beat the other?

TL;DR: GPTQ rounds weights one input column at a time and pushes each rounding error onto the columns not yet quantized, using the inverse Hessian of the layer's calibration inputs. AWQ never changes weight values beyond rounding: it scales salient input channels up before round-to-nearest, choosing one exponent per layer by grid search. GPTQ fits the calibration set harder, which wins in-domain and loses when traffic shifts; in a synthetic toy below its output error goes from 0.0094 in-distribution to 0.057 under a shift, against 0.014 for plain rounding.

How to approach it

State the shared objective first, because both methods minimize the same thing: the layer's output error $\lVert XW^\top - X\hat{W}^\top \rVert^2$ on calibration inputs $X$. They differ in what they let themselves change to minimize it. Then ask what the calibration data is and how close it is to production traffic, the bit width (4 or 3), and which kernel will run the result, since act-order changes the kernel.

A strong answer

GPTQ: error feedback with second-order weights. For one linear layer, the loss above is quadratic in the weights with Hessian $H = 2X^\top X$. GPTQ quantizes input column $j$, measures the rounding error, and updates every later column by the amount that best cancels it, weighted by the Cholesky factor of $H^{-1}$. The weights it finally rounds are not the trained weights; they were nudged to absorb earlier errors. The core loop, run against NumPy:

import numpy as np

def gptq_layer(W, X, bits=4, g=128, damp=0.01):
    """W: [out, in] weights, X: [samples, in] calibration inputs."""
    W = W.copy()
    H = X.T @ X                                    # [in, in], proxy Hessian
    H += damp * np.mean(np.diag(H)) * np.eye(len(H))
    U = np.linalg.cholesky(np.linalg.inv(H)).T     # upper factor of H^-1
    qmax = 2 ** (bits - 1) - 1
    Q = np.zeros_like(W)
    for j in range(W.shape[1]):                    # one input column at a time
        if j % g == 0:                             # new group: new scale
            scale = np.abs(W[:, j:j + g]).max(1) / qmax
        Q[:, j] = np.clip(np.round(W[:, j] / scale), -qmax - 1, qmax) * scale
        err = (W[:, j] - Q[:, j]) / U[j, j]
        W[:, j + 1:] -= np.outer(err, U[j, j + 1:])  # push error onto later columns
    return Q

Production implementations batch the updates over blocks of columns so the work runs as GEMMs; the arithmetic is the same. The cost is the Hessian, one per distinct input:

H is in x in, FP32. Llama 3.3 70B:
  q/k/v_proj input 8,192   : 8,192^2  x 4 B = 268 MB
  down_proj input 28,672   : 28,672^2 x 4 B = 3.29 GB
sanity: one layer at a time fits a single GPU, which is why GPTQ runs layer-wise

The GPTQ authors report quantizing 175B-parameter models in about four GPU hours (arXiv 2210.17323, abstract).

AWQ: protect the channels that carry large activations. AWQ's premise is that a small fraction of weights, those multiplying input channels with large activation magnitudes, dominate the output error. Scaling input channel $j$ of $W$ up by $s_j$ (and the activation down by the same factor, folded into the previous op) makes those weights larger relative to their group's rounding step, so their relative rounding error shrinks. AWQ sets $s_j = \bar{|x_j|}^{\alpha}$ and grid-searches the single exponent $\alpha$ per layer on calibration output error, then rounds to nearest. There is no reconstruction and no backprop, which the authors argue keeps it from overfitting the calibration set (arXiv 2306.00978).

What that difference does under shift. A toy layer, 256 inputs by 128 outputs, correlated Gaussian inputs with four outlier channels, INT4 in groups of 128. Calibrated on one sample, evaluated on a fresh sample from the same distribution and on one whose outliers moved to four other channels:

output MSE (synthetic toy, NumPy)   round-to-nearest   GPTQ     AWQ (alpha 0.25)
calibration set                     0.0131             0.0077   0.0058
held-out, same distribution         0.0133             0.0094   0.0059
held-out, outliers moved            0.0139             0.0572   0.0211

GPTQ's error feedback encodes the calibration covariance into the weights, so it gains in-domain and pays when the covariance changes. AWQ also loses under the shift (it protected the wrong channels) but less. This is a constructed example meant to show the mechanism, not a model benchmark.

The decision for a serving team:

• Calibration data matches traffic, 4-bit: either works. Run both and keep the one that wins on the served tasks; the output files and kernels are the same.
• Mixed or unknown traffic, or a small calibration set: AWQ, because its single exponent per layer has little capacity to overfit.
• 3-bit or very aggressive targets: GPTQ's compensation has more room to recover error than a scale alone. The two compose, since AWQ's scaling can be applied first and GPTQ's rounding after.

The reversal condition is calibration quality. With a large in-domain calibration set that includes the hard traffic (code, long context, other languages), GPTQ's tighter fit stops being a liability.

What interviewers probe next

"What is act-order and why do kernels care?" It quantizes columns in order of decreasing Hessian diagonal, so the most sensitive columns are rounded while the most correction budget remains. Group membership no longer follows memory order, so the kernel needs a group index per column or a reorder of the activations at load.

"What is the damping term for?" $X^\top X$ is often near-singular (dead or duplicated channels), and adding 1% of the mean diagonal keeps the Cholesky factorization stable.

"Does either change inference speed?" No, the format is the same INT4 plus group scales. Speed is decided by the kernel, apart from the act-order indexing above.

How it's scored

Weak: says "GPTQ is more accurate and AWQ is faster", or cannot say what either one changes in the weights.

Hire: describes GPTQ's column-by-column error compensation with the inverse Hessian and AWQ's activation-based channel scaling with round-to-nearest, and ties the choice to calibration data.

Strong Hire: explains why error feedback overfits the calibration covariance, sizes the Hessian for a real layer, knows act-order's kernel consequence, and picks per traffic mix with a reversal condition.

Common mistakes

• Claiming AWQ keeps 1% of weights in FP16. That was its motivating experiment; the method scales channels and every weight ends up INT4.
• Calibrating on the evaluation set, then reporting the in-domain win as general.
• Running GPTQ with act-order and a kernel that ignores the group index, which loads valid-looking weights with the wrong scales.
• Comparing the two at different group sizes and crediting the method for the group-size difference.