Affine Quantization vs Scale Quantization

Affine Quantization and Scale Quantization are range mapping techniques in Quantization (in ML). We have explored the difference between the two techniques (Affine Quantization vs Scale Quantization) in depth. The differences between Affine Quantization and Scale Quantization are as follows:

  1. INT8 range
    Int8 range in Affine Quantization is -128 to 127 while in Scale Quantization, it is -127 to 127. Note the missing -128 in Scale Quantization.

  2. Affine Quantization is asymmetric while Scale Quantization is symmetric.

  3. Quantization Equation

Affine Quantization uses:
F(x) = s.x + z

Scale Quantization uses:
F(x) = s.x
  1. Scale factor
Affine Quantization uses:
s = (2^B + 1)/(A1-A2)

Scale Quantization uses:
s = (2^(B - 1) − 1) / A1
  1. Zero Point
    Zero Point is used in Affine Quantization but it is not used in Scale Quantization.
Affine Quantization uses:
z = -(ROUND(A2 * s)) - 2^(B-1)

z = 0 (not used in Scale quantization)
  1. Dequantization
    The Dequantization equation is different in Affine and Scale quantization as scale quantization does not use the scale factor s.
Affine Quantization uses:
x_dequantize = (x_quantize − z) / s

Scale Quantization uses:
x_dequantize = x_quantize / s
  1. Scale Quantization makes the calculations easy and gives better accuracy for most models compared to Affine Quantization.

Quantization Equation

The Quantization Equation is different is both Affine Quantization and Scale Quantization. Affine Quantization uses the general Quantization equation while Scale Quantization uses the special case of the same equation.

Affine Quantization uses the following equation for Quantization:

F(x) = s.x + z

where

  • F is the quantization function
  • x is the input
  • F(x) is the quantized output
  • s is the scale factor
  • z is the zero point

The special case of the above equation is not used in Affine Quantization.

Scale Quantization uses the following equation for Quantization:

F(x) = s.x

Note that there is no z (zero point) in the equation.

Hence, the quantization equation is different in both cases (Affine Quantization vs Scale Quantization).

As there is no zero point in Scale Quantization, the equation for dequantization also changes.

The only drawback in Scale Quantization is that is uses one less value as its range compared to Affine Quantization but this symmetric nature makes the calculations easy and gives better accuracy for most models as well.

With this OPENGENUS article, you must have a strong idea of the differences with Affine Quantization and Scale Quantization.