PyTorch implementation of Data Free Quantization Through Weight Equalization and Bias Correction with some ideas from ZeroQ: A Novel Zero Shot Quantization Framework.

• Tested with MobileNetV2 and ResNet-18
• ImageNet validation set (Acc.)

• Tested with Deeplab-v3-plus_mobilenetv2

• Tested with MobileNetV2 SSD-Lite model

There are 6 arguments, all default to False

1. quantize: whether to quantize parameters and activations.
2. relu: whether to replace relu6 to relu.
3. equalize: whether to perform cross layer equalization.
4. correction: whether to apply bias correction
5. clip_weight: whether to clip weights in range [-15, 15] (for convolution and linear layer)
6. distill_range: whether to use distill data for setting min/max range of activation quantization

run the equalized model by:

python main_cls.py --quantize --relu --equalize

run the equalized and bias-corrected model by:

python main_cls.py --quantize --relu --equalize --correction

run the equalized and bias-corrected model with distilled data by:

python main_cls.py --quantize --relu --equalize --correction --distill_range

According to recent paper ZeroQ, we can distill some fake data to match the statistics from batch-normalization layers, then use it to set the min/max value range of activation quantization.
It does not need each conv followed by batch norm layer, and should produce better and more stable results using distilled data (the method from DFQ sometimes failed to find a good enough value range).

Here are some modifications that differs from original ZeroQ implementation:

1. Initialization of distilled data
2. Early stop criterion

Also, I think it can be applied to optimizing cross layer equalization and bias correction. The results will be updated as long as I make it to work.

The 'Int8' model in this repo is actually simulation of 8 bits, the actual calculation is done in floating points.
This is done by quantizing-dequantizing parameters in each layer and activation between 2 consecutive layers;
Which means each tensor will have dtype 'float32', but there would be at most 256 (2^8) unique values in it.

  Weight_quant(Int8) = Quant(Weight)
  Weight_quant(FP32) = Weight_quant(Int8*) = Dequant(Quant(Weight))

Somehow I cannot make Bias-Correction work on 8-bits bias quantization (even with data dependent correction).
I am not sure how the original paper managed to do it with 8 bits quantization, but I guess they either use some non-uniform quantization techniques or use more bits for bias parameters as I do.

• cross layer equalization
• high bias absorption
• data-free bias correction
• test with detection model
• test with classification model
• use distilled data to set min/max activation range
• use distilled data to find optimal scale matrix
• use distilled data to do bias correction
• True Int8 inference

• https://github.com/jfzhang95/pytorch-deeplab-xception
• https://github.com/ricky40403/PyTransformer
• https://github.com/qfgaohao/pytorch-ssd
• https://github.com/tonylins/pytorch-mobilenet-v2
• https://github.com/xxradon/PytorchToCaffe
• https://github.com/amirgholami/ZeroQ