Release v1.7.0 of NNCF to master

Author: vshampor

CONTRIBUTING.md

@@ -31,6 +31,8 @@ If your testing code is more extensive than unit tests (in terms of test executi
 
 
 ## Code style
+Changes to NNCF Python code should conform to [Python Style Guide](./docs/styleguide/PyGuide.md)
+
 Pylint is used throughout the project to ensure code cleanliness and quality. 
 A Pylint run is also done as part of the pre-commit scope - the pre-commit `pytest` scope will not be run if your code fails the Pylint checks.
 The Pylint rules and exceptions for this repository are described in the standard [.pylintrc](./.pylintrc) format - make sure your local linter uses these.

README.md

@@ -77,8 +77,16 @@ See [third_party_integration](./third_party_integration) for examples of code mo
 ## System requirements
 - Ubuntu\* 18.04 or later (64-bit)
 - Python\* 3.7 or later
-- NVidia CUDA\* Toolkit 10.2
-- PyTorch\* 1.7 or later.
+- NVidia CUDA\* Toolkit 10.2 or later^
+- PyTorch\* 1.5 or later (1.8.0 not supported, 1.8.1 supported)
+
+*NOTE:* The best known PyTorch version for the current NNCF is 1.8.1, and it is highly recommended to use it.
+
+^ If a torch package built for specific CUDA version is already present in the environment into which NNCF is being installed,
+and if it has a matching base version, then the CUDA version for which the present torch package is targeted will be used.
+
+Otherwise NNCF will install the latest available torch version from pip, which is targeted to the CUDA version of PyTorch packaging strategy's choosing.
+For PyTorch 1.8.1, the default CUDA is 10.2.
 
 ## Installation
 We suggest to install or use the package in the [Python virtual environment](https://docs.python.org/3/tutorial/venv.html).
@@ -91,10 +99,8 @@ We suggest to install or use the package in the [Python virtual environment](htt
 
 2) Install the package and its dependencies by running the following in the repository root directory:
 
-- For CPU & GPU-powered execution:
 `python setup.py install`
-- For CPU-only installation
-`python setup.py install --cpu-only`
+
 
 _NB_: For launching example scripts in this repository, we recommend replacing the `install` option above with `develop` and setting the `PYTHONPATH` variable to the root of the checked-out repository.
 
@@ -110,7 +116,7 @@ Use one of the Dockerfiles in the [docker](./docker) directory to build an image
 
 **NOTE**: If you want to use sample training scripts provided in the NNCF repository under `examples`, you should install the corresponding Python package dependencies:
 ```
-pip install examples/requirements.txt
+pip install -r examples/requirements.txt
 ```
 
 ## Contributing
@@ -139,46 +145,49 @@ Quick jump to sample type:
 
 |Model|Compression algorithm|Dataset|PyTorch FP32 baseline|PyTorch compressed accuracy|
 | :---: | :---: | :---: | :---: | :---: |
-|ResNet-50|INT8|ImageNet|76.04|75.97|
-|ResNet-50|Mixed, 44.8% INT8 / 55.2% INT4|ImageNet|76.04|76.31|
-|ResNet-50|INT8 + Sparsity 61% (RB)|ImageNet|76.04|75.21|
-|ResNet-50|INT8 + Sparsity 50% (RB)|ImageNet|76.04|75.73|
-|ResNet-50|Filter pruning, 40%, geometric median criterion|ImageNet|76.04|75.57|
-|Inception V3|INT8|ImageNet|77.31|76.97|
-|Inception V3|INT8 + Sparsity 61% (RB)|ImageNet|77.31|76.95|
-|MobileNet V2|INT8|ImageNet|71.75|71.32|
-|MobileNet V2|Mixed, 46.6% INT8 / 53.4% INT4|ImageNet|71.75|71.03|
-|MobileNet V2|INT8 + Sparsity 52% (RB)|ImageNet|71.75|70.94|
-|SqueezeNet V1.1|INT8|ImageNet|58.24|58.04|
-|SqueezeNet V1.1|Mixed, 54.7% INT8 / 45.3% INT4|ImageNet|58.24|58.77|
-|ResNet-18|XNOR (weights), scale/threshold (activations)|ImageNet|69.68|61.57|
-|ResNet-18|DoReFa (weights), scale/threshold (activations)|ImageNet|69.68|61.65|
-|ResNet-18|Filter pruning, 40%, magnitude criterion|ImageNet|69.68|69.19|
-|ResNet-18|Filter pruning, 40%, geometric median criterion|ImageNet|69.68|69.29|
-|ResNet-34|Filter pruning, 40%, geometric median criterion|ImageNet|73.26|72.72|
-|GoogLeNet|Filter pruning, 40%, geometric median criterion|ImageNet|69.72|68.89|
+|ResNet-50|INT8|ImageNet|76.16|76.42|
+|ResNet-50|INT8 (per-tensor only)|ImageNet|76.16|76.37|
+|ResNet-50|Mixed, 44.8% INT8 / 55.2% INT4|ImageNet|76.16|76.2|
+|ResNet-50|INT8 + Sparsity 61% (RB)|ImageNet|76.16|75.43|
+|ResNet-50|INT8 + Sparsity 50% (RB)|ImageNet|76.16|75.55|
+|ResNet-50|Filter pruning, 40%, geometric median criterion|ImageNet|76.16|75.62|
+|Inception V3|INT8|ImageNet|77.34|78.25|
+|Inception V3|INT8 + Sparsity 61% (RB)|ImageNet|77.34|77.58|
+|MobileNet V2|INT8|ImageNet|71.93|71.35|
+|MobileNet V2|INT8 (per-tensor only)|ImageNet|71.93|71.3|
+|MobileNet V2|Mixed, 46.6% INT8 / 53.4% INT4|ImageNet|71.93|70.92|
+|MobileNet V2|INT8 + Sparsity 52% (RB)|ImageNet|71.93|71.11|
+|SqueezeNet V1.1|INT8|ImageNet|58.24|58.28|
+|SqueezeNet V1.1|INT8 (per-tensor only)|ImageNet|58.24|58.26|
+|SqueezeNet V1.1|Mixed, 54.7% INT8 / 45.3% INT4|ImageNet|58.24|58.9|
+|ResNet-18|XNOR (weights), scale/threshold (activations)|ImageNet|69.8|61.63|
+|ResNet-18|DoReFa (weights), scale/threshold (activations)|ImageNet|69.8|61.61|
+|ResNet-18|Filter pruning, 40%, magnitude criterion|ImageNet|69.8|69.26|
+|ResNet-18|Filter pruning, 40%, geometric median criterion|ImageNet|69.8|69.32|
+|ResNet-34|Filter pruning, 40%, geometric median criterion|ImageNet|73.3|72.73|
+|GoogLeNet|Filter pruning, 40%, geometric median criterion|ImageNet|69.75|68.82|
 
 #### Object detection
 
 |Model|Compression algorithm|Dataset|PyTorch FP32 baseline|PyTorch compressed accuracy|
 | :---: | :---: | :---: | :---: | :---: |
-|SSD300-MobileNet|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC12 eval|86.02|85.82|
-|SSD300-BN|INT8|VOC12+07 train, VOC12 eval|96.53|96.5|
-|SSD300-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC12 eval|96.53|95.4|
-|SSD300-BN|Filter pruning, 40%, geometric median criterion|VOC12+07 train, VOC12 eval|96.53|91.98|
-|SSD512-BN|INT8|VOC12+07 train, VOC12 eval|97.59|97.37|
-|SSD512-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC12 eval|97.59|96.66|
+|SSD300-MobileNet|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|62.23|62.94|
+|SSD300-VGG-BN|INT8|VOC12+07 train, VOC07 eval|78.28|77.96|
+|SSD300-VGG-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|78.28|77.59|
+|SSD300-VGG-BN|Filter pruning, 40%, geometric median criterion|VOC12+07 train, VOC07 eval|78.28|77.72|
+|SSD512-VGG-BN|INT8|VOC12+07 train, VOC07 eval|80.26|80.12|
+|SSD512-VGG-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|80.26|79.67|
 
 #### Semantic segmentation
 
 |Model|Compression algorithm|Dataset|PyTorch FP32 baseline|PyTorch compressed accuracy|
 | :---: | :---: | :---: | :---: | :---: |
-|UNet|INT8|CamVid|71.95|71.66|
-|UNet|INT8 + Sparsity 60% (Magnitude)|CamVid|71.95|71.72|
-|ICNet|INT8|CamVid|67.89|67.78|
-|ICNet|INT8 + Sparsity 60% (Magnitude)|CamVid|67.89|67.16|
-|UNet|INT8|Mapillary|56.23|56.1|
-|UNet|INT8 + Sparsity 60% (Magnitude)|Mapillary|56.23|56.01|
+|UNet|INT8|CamVid|71.95|71.8|
+|UNet|INT8 + Sparsity 60% (Magnitude)|CamVid|71.95|72.03|
+|ICNet|INT8|CamVid|67.89|67.86|
+|ICNet|INT8 + Sparsity 60% (Magnitude)|CamVid|67.89|67.18|
+|UNet|INT8|Mapillary|56.23|55.87|
+|UNet|INT8 + Sparsity 60% (Magnitude)|Mapillary|56.23|55.65|
 |UNet|Filter pruning, 25%, geometric median criterion|Mapillary|56.23|55.62|
 
 

ReleaseNotes.md

@@ -7,6 +7,28 @@ samples distributed with the code.  The samples demonstrate the usage of compres
 public models and datasets for three different use cases: Image Classification, Object Detection,
 and Semantic Segmentation.
 
+## New in Release 1.7:
+- Adjust Padding feature to support accurate execution of U4 on VPU - when setting "target_device" to "VPU", the training-time padding values for quantized convolutions will be adjusted to better reflect VPU inference process.
+- Weighted layers that are "frozen" (i.e. have requires_grad set to False at compressed model creation time) are no longer considered for compression, to better handle transfer learning cases.
+- Quantization algorithm now sets up quantizers without giving an option for requantization, which guarantees best performance, although at some cost to quantizer configuration flexibility.
+- Pruning models with FCOS detection heads and instance normalization operations now supported
+- Added a mean percentile initializer for the quantization algorithm
+- Now possible to additionally quantize model outputs (separate control for each output quantization is supported)
+- Models quantized for CPU now use effective 7-bit quantization for weights - the ONNX-exported model is still configured to use 8 bits for quantization, but only the middle 128 quanta of the total possible 256 are actually used, which allows for better OpenVINO inference accuracy alignment with PyTorch on non-VNNI CPUs
+- Bumped target PyTorch version to 1.8.1 and relaxed package requirements constraints to allow installation into environments with PyTorch >=1.5.0
+
+Notable bugfixes:
+- Fixed bias pruning in depthwise convolution
+- Made per-tensor quantization available for all operations that support per-channel quantization 
+- Fixed progressive training performance degradation when an output tensor of an NNCF-compressed model is reused as its input.
+- `pip install .` path of installing NNCF from a checked-out repository is now supported.
+- Nested `with no_nncf_trace()` blocks now function as expected.
+- NNCF compression API now formally abstract to guard against virtual function calls
+- Now possible to load AutoQ and HAWQ-produced checkpoints to evaluate them or export to ONNX
+
+Removed features:
+- Pattern-based quantizer setup mode for quantization algorithm - due to its logic, it did not guarantee that all required operation inputs are ultimately quantized.
+
 
 ## New in Release 1.6:
 - Added AutoQ - an AutoML-based mixed-precision initialization mode for quantization, which utilizes the power of reinforcement learning to select the best quantizer configuration for any model in terms of quality metric for a given HW architecture type.

beta/README.md

@@ -90,14 +90,16 @@ Quick jump to the samples:
 |ResNet50|INT8 w:sym,per-tensor a:sym,per-tensor|ImageNet|75.04|75.04|
 |ResNet50|Sparsity 50% (Magnitude)|ImageNet|75.04|75|
 |ResNet50|INT8 w:sym,per-tensor a:sym,per-tensor + Sparsity 50% (Magnitude)|ImageNet|75.04|74.46|
-|TensorFlow Hub MobileNet V2|Sparsity 35% (Magnitude)|ImageNet|71.84|71.73|
+|ResNet50|Filter Pruning 40%|ImageNet|75.04|74.98|
+|TensorFlow Hub MobileNet V2|Sparsity 35% (Magnitude)|ImageNet|71.84|71.90|
 
 #### Object detection
 
 |**Model**|**Compression algorithm**|**Dataset**|**TensorFlow FP32 baseline mAP**|**TensorFlow compressed mAP**|
 | :---: | :---: | :---: | :---: | :---: |
 |RetinaNet|INT8 w:sym,per-tensor a:sym,per-tensor |COCO2017|33.44|33.3|
 |RetinaNet|Sparsity 50% (Magnitude)|COCO2017|33.44|33.13|
+|RetinaNet|Filter Pruning 40%|COCO2017|33.44|32.7|
 
 #### Instance Segmentation
 

beta/examples/tensorflow/classification/README.md

@@ -111,4 +111,5 @@ To export a model to the OpenVINO IR and run it using the Intel® Deep Learning
 |ResNet50|INT8 w:sym,per-tensor a:sym,per-tensor|ImageNet|75.04|[resnet50_imagenet_int8.json](configs/quantization/resnet50_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_int8_w_sym_t_a_sym_t.tar.gz)|
 |ResNet50|Sparsity 50% (Magnitude)|ImageNet|75|[resnet50_imagenet_magnitude_sparsity.json](configs/sparsity/resnet50_imagenet_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_sparsity_50.tar.gz)|
 |ResNet50|INT8 w:sym,per-tensor a:sym,per-tensor + Sparsity 50% (Magnitude)|ImageNet|74.46|[resnet50_imagenet_magnitude_sparsity_int8.json](configs/sparsity_quantization/resnet50_imagenet_magnitude_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_int8_w_sym_t_a_sym_t_sparsity_50.tar.gz)|
-|TensorFlow Hub MobileNet V2|Sparsity 35% (Magnitude)|ImageNet|71.73|[mobilenet_v2_hub_imagenet_magnitude_sparsity.json](configs/sparsity/mobilenet_v2_hub_imagenet_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/tf1_mobilenet_v2_1.0_224_s0.35.tar.gz)|
+|ResNet50|Filter Pruning 40%|ImageNet|74.98|[resnet50_imagenet_pruning_geometric_median.json](configs/pruning/resnet50_imagenet_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_pruning_40.tar.gz)|
+|TensorFlow Hub MobileNet V2|Sparsity 35% (Magnitude)|ImageNet|71.90|[mobilenet_v2_hub_imagenet_magnitude_sparsity.json](configs/sparsity/mobilenet_v2_hub_imagenet_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/tf1_mobilenet_v2_1.0_224_s0.35.tar.gz)|

beta/examples/tensorflow/classification/configs/pruning/resnet50_imagenet_pruning_geometric_median.json

@@ -0,0 +1,35 @@
+{
+    "model": "ResNet50",
+    "pretrained": true,
+    "input_info": {
+        "sample_size": [1, 224, 224, 3]
+    },
+    "batch_size": 320,
+    "epochs": 65,
+    "optimizer": {
+        "type": "sgd",
+        "schedule_type": "multistep",
+        "schedule_params": {
+            "base_lr": 0.1,
+            "steps": [20, 40, 60],
+            "gamma": 0.1
+        },
+        "optimizer_params": {
+            "momentum": 0.9,
+            "nesterov": true
+        }
+    },
+    "dataset": "imagenet2012",
+    "dataset_type": "tfds",
+
+    "compression": {
+        "algorithm": "filter_pruning",
+        "pruning_init": 0.1,
+        "params": {
+            "schedule": "exponential",
+            "pruning_target": 0.4,
+            "pruning_steps": 15,
+            "weight_importance": "geometric_median"
+        }
+    }
+}

beta/examples/tensorflow/classification/configs/quantization/inception_v3_imagenet_int8.json

@@ -9,7 +9,7 @@
     "epochs": 10,
 
     "optimizer": {
-        "type": "adam",
+        "type": "Adam",
         "schedule_type": "piecewise_constant",
         "schedule_params": {
             "boundaries": [5],

beta/examples/tensorflow/classification/configs/quantization/mobilenet_v2_imagenet_int8.json

@@ -9,7 +9,7 @@
     "epochs": 15,
 
     "optimizer": {
-        "type": "adam",
+        "type": "Adam",
         "schedule_type": "piecewise_constant",
         "schedule_params": {
             "boundaries": [5, 10],

beta/examples/tensorflow/classification/configs/quantization/resnet50_imagenet_int8.json

@@ -7,7 +7,7 @@
     "batch_size": 128,
     "epochs": 4,
     "optimizer": {
-        "type": "adam",
+        "type": "Adam",
         "schedule_type": "piecewise_constant",
         "schedule_params": {
             "boundaries": [2],

beta/examples/tensorflow/classification/configs/sparsity/inception_v3_imagenet_magnitude_sparsity.json

@@ -9,7 +9,7 @@
     "epochs": 40,
 
      "optimizer": {
-        "type": "adam",
+        "type": "Adam",
         "schedule_type": "piecewise_constant",
         "schedule_params": {
             "boundaries": [20, 25, 30],