diff --git a/README.md b/README.md index 6b898678e..a5065aac1 100644 --- a/README.md +++ b/README.md @@ -1,66 +1,101 @@ -# Model Compression Toolkit (MCT) - -Model Compression Toolkit (MCT) is an open-source project for neural network model optimization under efficient, constrained hardware. +
-This project provides researchers, developers, and engineers tools for optimizing and deploying state-of-the-art neural networks on efficient hardware. - -Specifically, this project aims to apply quantization to compress neural networks. +# Model Compression Toolkit (MCT) - +**Open-source project for neural network model optimization, providing researchers, developers, and engineers with advanced quantization and compression tools for deploying state-of-the-art neural networks on efficient, constrained hardware.** +______________________________________________________________________ + +
+
+

+ Getting Started • + Tutorials • + High level features and techniques • + Resources • + Community • + License +

+

+ + + + + + +

+
+ +__________________________________________________________________________________________________________ -MCT is developed by researchers and engineers working at Sony Semiconductor Israel. +## Getting Started +### Quick Installation +Pip install the model compression toolkit package in a Python>=3.9 environment with PyTorch>=2.1 or Tensorflow>=2.12. +``` +pip install model-compression-toolkit +``` +For installing the nightly version or installing from source, refer to the [installation guide](https://github.com/sony/model_optimization/blob/main/INSTALLATION.md). +**Important note**: In order to use MCT, you’ll need to provide a floating point .pt or .keras model as an input. +### Tutorials and Examples -## Table of Contents +Our [tutorials](https://github.com/sony/model_optimization/blob/main/tutorials/README.md) section will walk you through the basics of the MCT tool, covering various compression techniques for both Keras and PyTorch models. +Access interactive notebooks for hands-on learning with popular models/tasks or move on to [Resources](#resources) section. -- [Getting Started](https://github.com/sony/model_optimization?tab=readme-ov-file#getting-started) -- [Supported features](https://github.com/sony/model_optimization?tab=readme-ov-file#supported-features) -- [Results](https://github.com/sony/model_optimization?tab=readme-ov-file#results) -- [Troubleshooting](https://github.com/sony/model_optimization?tab=readme-ov-file#trouble-shooting) -- [Contributions](https://github.com/sony/model_optimization?tab=readme-ov-file#contributions) -- [License](https://github.com/sony/model_optimization?tab=readme-ov-file#license) +### Supported Quantization Methods +MCT supports various quantization methods as appears below. +
+

+ Quantization Method | Complexity | Computational Cost | Tutorial +-------------------- | -----------|--------------------|--------- +PTQ (Post Training Quantization) | Low | Low (~1-10 CPU minutes) | +GPTQ (parameters fine-tuning using gradients) | Moderate | Moderate (~1-3 GPU hours) | +QAT (Quantization Aware Training) | High | High (~12-36 GPU hours) | -## Getting Started +

+
-This section provides an installation and a quick starting guide. +For each flow, **Quantization core** utilizes various algorithms and hyper-parameters for optimal [hardware-aware](https://github.com/sony/model_optimization/blob/main/model_compression_toolkit/target_platform_capabilities/README.md) quantization results. +For further details, please see [Supported features and algorithms](#supported-features). -### Installation +Required input: +- Floating point model - 32bit model in either .pt or .keras format +- Representative dataset - can be either provided by the user, or generated utilizing the [Data Generation](#data-generation-) capability -To install the latest stable release of MCT, run the following command: -``` -pip install model-compression-toolkit -``` +
+

-For installing the nightly version or installing from source, refer to the [installation guide](https://github.com/sony/model_optimization/blob/main/INSTALLATION.md). + +

+
+### Resources +* [User Guide](https://sony.github.io/model_optimization/docs/index.html) contains detailed information about MCT and guides you from installation through optimizing models for your edge AI applications. -### Quick start & tutorials +* MCT's [API Docs](https://sony.github.io/model_optimization/docs/api/api_docs/) is seperated per quantization methods: -Explore the Model Compression Toolkit (MCT) through our tutorials, -covering compression techniques for Keras and PyTorch models. Access interactive [notebooks](https://github.com/sony/model_optimization/blob/main/tutorials/README.md) -for hands-on learning. For example: -* [Keras MobileNetV2 post training quantization](https://github.com/sony/model_optimization/blob/main/tutorials/notebooks/imx500_notebooks/keras/example_keras_mobilenetv2_for_imx500.ipynb) -* [Post training quantization with PyTorch](https://github.com/sony/model_optimization/blob/main/tutorials/notebooks/mct_features_notebooks/pytorch/example_pytorch_post_training_quantization.ipynb) -* [Data Generation for ResNet18 with PyTorch](https://github.com/sony/model_optimization/blob/main/tutorials/notebooks/mct_features_notebooks/pytorch/example_pytorch_data_generation.ipynb). + * [Post-training quantization](https://sony.github.io/model_optimization/docs/api/api_docs/index.html#ptq) | PTQ API docs + * [Gradient-based post-training quantization](https://sony.github.io/model_optimization/docs/api/api_docs/index.html#gptq) | GPTQ API docs + * [Quantization-aware training](https://sony.github.io/model_optimization/docs/api/api_docs/index.html#qat) | QAT API docs + +* [Debug](https://sony.github.io/model_optimization/docs/guidelines/visualization.html) – modify optimization process or generate explainable report + +* [Release notes](https://github.com/sony/model_optimization/releases) ### Supported Versions Currently, MCT is being tested on various Python, Pytorch and TensorFlow versions: +
+ Supported Versions Table - -| | PyTorch 2.1 | PyTorch 2.2 | PyTorch 2.3 | PyTorch 2.4 | PyTorch 2.5 | -|-------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Python 3.9 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch21.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch21.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch22.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch22.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch23.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch23.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch24.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch24.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch25.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch25.yml) | -| Python 3.10 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch21.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch21.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch22.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch22.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch23.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch23.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch24.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch24.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch25.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch25.yml) | -| Python 3.11 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch21.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch21.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch22.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch22.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch23.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch23.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch24.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch24.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch25.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch25.yml) | -| Python 3.12 | | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch22.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch22.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch23.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch23.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch24.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch24.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch25.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch25.yml) | - - - - +| | PyTorch 2.2 | PyTorch 2.3 | PyTorch 2.4 | PyTorch 2.5 | +|-------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Python 3.9 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch22.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch22.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch23.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch23.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch24.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch24.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch25.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch25.yml) | +| Python 3.10 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch22.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch22.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch23.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch23.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch24.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch24.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch25.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch25.yml) | +| Python 3.11 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch22.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch22.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch23.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch23.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch24.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch24.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch25.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch25.yml) | +| Python 3.12 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch22.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch22.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch23.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch23.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch24.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch24.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch25.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch25.yml) | | | TensorFlow 2.12 | TensorFlow 2.13 | TensorFlow 2.14 | TensorFlow 2.15 | |-------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| @@ -68,6 +103,7 @@ Currently, MCT is being tested on various Python, Pytorch and TensorFlow version | Python 3.10 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras212.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras212.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras213.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras213.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras214.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras214.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras215.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras215.yml) | | Python 3.11 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras212.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras212.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras213.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras213.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras214.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras214.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras215.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras215.yml) | +
## Supported Features MCT offers a range of powerful features to optimize neural network models for efficient deployment. These supported features include: @@ -85,9 +121,9 @@ MCT supports different quantization methods: | Quantization Method | Complexity | Computational Cost | |-----------------------------------------------|------------|-----------------------------| -| PTQ | Low | Low (order of minutes) | -| GPTQ (parameters fine-tuning using gradients) | Mild | Mild (order of 2-3 hours) | -| QAT | High | High (order of 12-36 hours) | +| PTQ | Low | Low (~CPU minutes) | +| GPTQ (parameters fine-tuning using gradients) | Moderate | Moderate (~1-3 GPU hours) | +| QAT | High | High (~12-36 GPU hours) | In addition, MCT supports different quantization schemes for quantizing weights and activations: @@ -133,15 +169,14 @@ For more details, we highly recommend visiting our project website where experim ## Results -### Keras -Graph of [MobileNetV2](https://keras.io/api/applications/mobilenet/) accuracy on ImageNet vs average bit-width of weights, using -single-precision quantization, mixed-precision quantization, and mixed-precision quantization with GPTQ. - - -For more results, please see [1] +

+ + + + -### Pytorch + ### Pytorch We quantized classification networks from the torchvision library. In the following table we present the ImageNet validation results for these models: @@ -151,6 +186,14 @@ In the following table we present the ImageNet validation results for these mode | ResNet-18 [3] | 69.86 | 69.63 |69.53| | SqueezeNet 1.1 [3] | 58.128 | 57.678 || +### Keras +MCT can quantize an existing 32-bit floating-point model to an 8-bit fixed-point (or less) model without compromising accuracy. +Below is a graph of [MobileNetV2](https://keras.io/api/applications/mobilenet/) accuracy on ImageNet vs average bit-width of weights (X-axis), using +single-precision quantization, mixed-precision quantization, and mixed-precision quantization with GPTQ. + + + +For more results, please see [1] #### Pruning Results @@ -161,23 +204,26 @@ Results for applying pruning to reduce the parameters of the following models by | ResNet50 [2] | 75.1 | 72.4 | | DenseNet121 [3] | 74.44 | 71.71 | +## Troubleshooting and Community -## Trouble Shooting +If you encountered large accuracy degradation with MCT, check out the [Quantization Troubleshooting](https://github.com/sony/model_optimization/tree/main/quantization_troubleshooting.md) +for common pitfalls and some tools to improve quantized model's accuracy. -If the accuracy degradation of the quantized model is too large for your application, check out the [Quantization Troubleshooting](https://github.com/sony/model_optimization/tree/main/quantization_troubleshooting.md) -for common pitfalls and some tools to improve quantization accuracy. +Check out the [FAQ](https://github.com/sony/model_optimization/tree/main/FAQ.md) for common issues. -Check out the [FAQ](https://github.com/sony/model_optimization/tree/main/FAQ.md) for common issues. +You are welcome to ask questions and get support on our [issues section](https://github.com/sony/model_optimization/issues) and manage community discussions under [discussions section](https://github.com/sony/model_optimization/discussions). ## Contributions MCT aims at keeping a more up-to-date fork and welcomes contributions from anyone. -*You will find more information about contributions in the [Contribution guide](https://github.com/sony/model_optimization/blob/main/CONTRIBUTING.md). +*Checkout our [Contribution guide](https://github.com/sony/model_optimization/blob/main/CONTRIBUTING.md) for more details. ## License -[Apache License 2.0](https://github.com/sony/model_optimization/blob/main/LICENSE.md). +MCT is licensed under Apache License Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms. + + ## References diff --git a/docsrc/images/Classification.png b/docsrc/images/Classification.png new file mode 100644 index 000000000..c5359baf4 Binary files /dev/null and b/docsrc/images/Classification.png differ diff --git a/docsrc/images/ObjDet.png b/docsrc/images/ObjDet.png new file mode 100644 index 000000000..a92b29c1e Binary files /dev/null and b/docsrc/images/ObjDet.png differ diff --git a/docsrc/images/PoseEst.png b/docsrc/images/PoseEst.png new file mode 100644 index 000000000..2f342432e Binary files /dev/null and b/docsrc/images/PoseEst.png differ diff --git a/docsrc/images/SemSeg.png b/docsrc/images/SemSeg.png new file mode 100644 index 000000000..ddb2e4027 Binary files /dev/null and b/docsrc/images/SemSeg.png differ diff --git a/docsrc/images/colab-badge.svg b/docsrc/images/colab-badge.svg new file mode 100644 index 000000000..e5830d533 --- /dev/null +++ b/docsrc/images/colab-badge.svg @@ -0,0 +1 @@ + Open in ColabOpen in Colab diff --git a/docsrc/images/mctDiagramFlow.svg b/docsrc/images/mctDiagramFlow.svg new file mode 100644 index 000000000..ae8f33635 --- /dev/null +++ b/docsrc/images/mctDiagramFlow.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docsrc/images/mctDiagram_clean.svg b/docsrc/images/mctDiagram_clean.svg new file mode 100644 index 000000000..cf983ea37 --- /dev/null +++ b/docsrc/images/mctDiagram_clean.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docsrc/images/mctFlow.png b/docsrc/images/mctFlow.png new file mode 100644 index 000000000..58246dd30 Binary files /dev/null and b/docsrc/images/mctFlow.png differ