From 46b5a878709cf0a13ca3eef04bb3623aa486ec09 Mon Sep 17 00:00:00 2001 From: Sumaiya Tarannum Noor Date: Wed, 15 Jan 2025 14:36:50 +0600 Subject: [PATCH] Add files via upload --- .../example_pytorch_data_generation.ipynb | 490 ++++++++++++++++++ 1 file changed, 490 insertions(+) create mode 100644 tutorials/notebooks/mct_features_notebooks/pytorch/example_pytorch_data_generation.ipynb diff --git a/tutorials/notebooks/mct_features_notebooks/pytorch/example_pytorch_data_generation.ipynb b/tutorials/notebooks/mct_features_notebooks/pytorch/example_pytorch_data_generation.ipynb new file mode 100644 index 000000000..98cfe4458 --- /dev/null +++ b/tutorials/notebooks/mct_features_notebooks/pytorch/example_pytorch_data_generation.ipynb @@ -0,0 +1,490 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "source": [ + "# Data Generation Tutorial: Data-Free (Zero-Shot) Quantization in Pytorch with the Model Compression Toolkit (MCT)\n", + "[Run this tutorial in Google Colab](https://colab.research.google.com/github/sony/model_optimization/blob/main/tutorials/notebooks/mct_features_notebooks/pytorch/example_pytorch_data_generation.ipynb)\n", + "\n", + "## Overview\n", + "In this tutorial, we will explore how to generate synthetic images using the Model Compression Toolkit (MCT) and the Data Generation Library. These generated images are based on the statistics stored in the model's batch normalization layers and can be usefull for various compression tasks, such as quantization and pruning. We will use the generated images as a representative dataset to quantize our model to 8-bit using MCT's Post Training Quantization (PTQ).\n", + "\n", + "## Summary\n", + "We will cover the following steps:\n", + "1. **Setup** Install and import necessary libraries and load a pre-trained model.\n", + "2. **Configuration**: Define the data generation configuration.\n", + "3. **Data Generation**: Generate synthetic images.\n", + "4. **Visualization**: Visualize the generated images.\n", + "5. **Quantization**: Quantize our model to 8-bit using PTQ with the generated images as a representative dataset. This is called **\"Data-Free Quantization\"** since no real data is used in the quantization process.\n", + "\n", + "## Step 1: Setup\n", + "Install the necessary packages:" + ], + "metadata": { + "collapsed": false, + "id": "74a56f1fe3c17fcf" + }, + "id": "74a56f1fe3c17fcf" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "!pip install -q torch torchvision" + ], + "metadata": { + "id": "941089a3a8cbdf3b" + }, + "id": "941089a3a8cbdf3b" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "import importlib\n", + "if not importlib.util.find_spec('model_compression_toolkit'):\n", + " !pip install model_compression_toolkit" + ], + "metadata": { + "id": "a0d8806b6aa0630a" + }, + "id": "a0d8806b6aa0630a" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "import torch\n", + "from torchvision.models import resnet18, ResNet18_Weights\n", + "from torchvision.datasets import ImageNet\n", + "from torch.utils.data import DataLoader\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np" + ], + "metadata": { + "id": "879b3f41ba5f6921" + }, + "id": "879b3f41ba5f6921" + }, + { + "cell_type": "markdown", + "source": [ + "Load the model from the torchvision library:" + ], + "metadata": { + "collapsed": false, + "id": "a6a2a1f3e024127" + }, + "id": "a6a2a1f3e024127" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "# Load a pre-trained model (e.g., ResNet18)\n", + "weights = ResNet18_Weights.DEFAULT\n", + "float_model = resnet18(weights=weights)" + ], + "metadata": { + "id": "2b0e10d5f3471530" + }, + "id": "2b0e10d5f3471530" + }, + { + "cell_type": "markdown", + "source": [ + "## Step 2: Define a Data Generation Configuration\n", + "Next, we need to specify the configuration for data generation using `get_pytorch_data_generation_config`. This configuration includes parameters such as the number of iterations, optimizer, batch size, and more. Customize these parameters according to your needs." + ], + "metadata": { + "collapsed": false, + "id": "2058e5c7f24c10f1" + }, + "id": "2058e5c7f24c10f1" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "import model_compression_toolkit as mct\n", + "\n", + "data_gen_config = mct.data_generation.get_pytorch_data_generation_config(\n", + " n_iter=500, # Number of iterations\n", + " optimizer=torch.optim.RAdam, # Optimizer\n", + " data_gen_batch_size=128, # Batch size for data generation\n", + " initial_lr=16, # Initial learning rate\n", + " output_loss_multiplier=1e-6, # Multiplier for output loss\n", + " extra_pixels=32,\n", + " # ... (customize other parameters)\n", + ")" + ], + "metadata": { + "id": "833dcf17ef4b49ac" + }, + "id": "833dcf17ef4b49ac" + }, + { + "cell_type": "markdown", + "source": [ + "## Step 3: Generate Synthetic Images\n", + "\n", + "Now, let's generate synthetic images using the `pytorch_data_generation_experimental` function. Specify the number of images you want to generate and the output image size." + ], + "metadata": { + "collapsed": false, + "id": "4e5fde47532e18fa" + }, + "id": "4e5fde47532e18fa" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "n_images = 256 # Number of images to generate\n", + "output_image_size = 224 # Size of output images\n", + "\n", + "generated_images = mct.data_generation.pytorch_data_generation_experimental(\n", + " model=float_model,\n", + " n_images=n_images,\n", + " output_image_size=output_image_size,\n", + " data_generation_config=data_gen_config\n", + ")" + ], + "metadata": { + "id": "606e623b954db7c5" + }, + "id": "606e623b954db7c5" + }, + { + "cell_type": "markdown", + "source": [ + "## Step 4: Visualization\n", + "Lets define a function to display the generated images:" + ], + "metadata": { + "collapsed": false, + "id": "831e12849e9962a4" + }, + "id": "831e12849e9962a4" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "def plot_image(image, reverse_preprocess=False, mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]):\n", + " image = image.detach().cpu().numpy()[0]\n", + " image = image.transpose(1, 2, 0)\n", + " if reverse_preprocess:\n", + " new_image = np.round(((image.astype(np.float32) * std) + mean) * 255).astype(np.uint8)\n", + " plt.imshow(new_image)\n", + " plt.show()" + ], + "metadata": { + "id": "efda94fc010113e3" + }, + "id": "efda94fc010113e3" + }, + { + "cell_type": "markdown", + "source": [ + "Now, let's visualize the generated images by selecting an image index to plot. You can modify the index values to experiment with different images." + ], + "metadata": { + "collapsed": false, + "id": "22c54ff3d5b2ad" + }, + "id": "22c54ff3d5b2ad" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "img_index_to_plot = 0\n", + "plot_image(generated_images[img_index_to_plot],True)" + ], + "metadata": { + "id": "e7da0f42acc69e20" + }, + "id": "e7da0f42acc69e20" + }, + { + "cell_type": "markdown", + "source": [ + "## Step 5: Post Training Quantization\n", + "In order to evaulate our generated images, we will use them to quantize the model using MCT's PTQ.This is referred to as **\"Zero-Shot Quantization (ZSQ)\"** or **\"Data-Free Quantization\"** because no real data is used in the quantization process. Next we will define configurations for MCT's PTQ.\n", + "\n", + "### Target Platform Capabilities (TPC)\n", + "MCT optimizes the model for dedicated hardware platforms. This is done using TPC (for more details, please visit our [documentation](https://sony.github.io/model_optimization/api/api_docs/modules/target_platform.html)). Here, we use the default Pytorch TPC:" + ], + "metadata": { + "collapsed": false, + "id": "7b40f70b4132c5fb" + }, + "id": "7b40f70b4132c5fb" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "target_platform_cap = mct.get_target_platform_capabilities(\"pytorch\", \"default\")" + ], + "metadata": { + "id": "672ffbf357234def" + }, + "id": "672ffbf357234def" + }, + { + "cell_type": "markdown", + "source": [ + "### Representative Dataset\n", + "For quantization with MCT, we need to define a representative dataset required by the PTQ algorithm. This dataset is a generator that returns a list of images. We wil use our generated images for the representative dataset." + ], + "metadata": { + "collapsed": false, + "id": "97073eeea51b4dee" + }, + "id": "97073eeea51b4dee" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "batch_size = 64\n", + "n_iter = 10\n", + "\n", + "generated_images = np.concatenate(generated_images, axis=0).reshape(*(-1, batch_size, *list(generated_images[0].shape[1:])))\n", + "\n", + "def representative_data_gen():\n", + " for nn in range(n_iter):\n", + " nn_mod = nn % generated_images.shape[0]\n", + " yield [generated_images[nn_mod]]" + ], + "metadata": { + "id": "d6a3d88a51883757" + }, + "id": "d6a3d88a51883757" + }, + { + "cell_type": "markdown", + "source": [ + "### Quantization with our generated images\n", + "Now, we are ready to use MCT to quantize the model." + ], + "metadata": { + "collapsed": false, + "id": "8cbc59406d217273" + }, + "id": "8cbc59406d217273" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "import torch\n", + "import copy\n", + "\n", + "# run post training quantization on the model to get the quantized model output\n", + "with torch.no_grad():\n", + " float_model_copy = resnet18(weights=weights) # Create a new model with the same architecture as float_model\n", + " float_model_copy.load_state_dict(float_model.state_dict()) # Copy the weights and biases\n", + "\n", + "quantized_model_generated_data, quantization_info = mct.ptq.pytorch_post_training_quantization(\n", + " in_module=float_model_copy,\n", + " representative_data_gen=representative_data_gen,\n", + " target_platform_capabilities=target_platform_cap\n", + ")" + ], + "metadata": { + "id": "c7f57ae27466992e" + }, + "id": "c7f57ae27466992e" + }, + { + "cell_type": "markdown", + "source": [ + "## Setup for evaluation on the ImageNet dataset\n", + "### Download ImageNet validation set\n", + "Download ImageNet dataset with only the validation split. This step may take several minutes..." + ], + "metadata": { + "collapsed": false, + "id": "1de293d52f60801" + }, + "id": "1de293d52f60801" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "import os\n", + "\n", + "if not os.path.isdir('imagenet'):\n", + " !mkdir imagenet\n", + " !wget -P imagenet https://image-net.org/data/ILSVRC/2012/ILSVRC2012_devkit_t12.tar.gz\n", + " !wget -P imagenet https://image-net.org/data/ILSVRC/2012/ILSVRC2012_img_val.tar\n", + "\n", + "# Extract ImageNet validation dataset using torchvision \"datasets\" module.\n", + "dataset = ImageNet(root='./imagenet', split='val', transform=weights.transforms())\n", + "val_dataloader = DataLoader(dataset, batch_size=50, shuffle=False, num_workers=16, pin_memory=True)" + ], + "metadata": { + "id": "5febfa57873fa2f3" + }, + "id": "5febfa57873fa2f3" + }, + { + "cell_type": "markdown", + "source": [ + "Here we define functions for evaluation:" + ], + "metadata": { + "collapsed": false, + "id": "874d5d61f876bc82" + }, + "id": "874d5d61f876bc82" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "from tqdm import tqdm\n", + "\n", + "\n", + "def evaluate(model, testloader):\n", + " device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n", + " model.to(device)\n", + " model.eval() # Set the model to evaluation mode\n", + " correct = 0\n", + " total = 0\n", + " with torch.no_grad():\n", + " for data in tqdm(testloader):\n", + " images, labels = data\n", + " images, labels = images.to(device), labels.to(device)\n", + " outputs = model(images)\n", + " _, predicted = outputs.max(1)\n", + " total += labels.size(0)\n", + " correct += predicted.eq(labels).sum().item()\n", + "\n", + " # correct += (predicted == labels).sum().item()\n", + " val_acc = (100 * correct / total)\n", + " print('Accuracy: %.2f%%' % val_acc)\n", + " return val_acc" + ], + "metadata": { + "id": "de8cdf0ada297905" + }, + "id": "de8cdf0ada297905" + }, + { + "cell_type": "markdown", + "source": [ + "### Evaluation of the quantized model's performance\n", + "Here we evaluate our model's top 1 classification performance after PTQ on the ImageNet validation dataset.\n", + "Let's start with the floating-point model evaluation." + ], + "metadata": { + "collapsed": false, + "id": "8cb543dcc791ee24" + }, + "id": "8cb543dcc791ee24" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "evaluate(float_model, val_dataloader)" + ], + "metadata": { + "id": "857b5d4111a42071" + }, + "id": "857b5d4111a42071" + }, + { + "cell_type": "markdown", + "source": [ + "Finally, let's evaluate the quantized model:" + ], + "metadata": { + "collapsed": false, + "id": "7451953d684a8497" + }, + "id": "7451953d684a8497" + }, + { + "cell_type": "code", + "execution_count": null, + "outputs": [], + "source": [ + "evaluate(quantized_model_generated_data, val_dataloader)" + ], + "metadata": { + "id": "e77e131927a14217" + }, + "id": "e77e131927a14217" + }, + { + "cell_type": "markdown", + "source": [ + "## Conclusion:\n", + "In this tutorial, we demonstrated how to generate synthetic images from a trained model and use them for model quantization. The quantized model achieved a 4x reduction in size compared to the original float model, while maintaining performance similar to the reported float results. Notably, no real data was required in this process." + ], + "metadata": { + "collapsed": false, + "id": "902909871aff0db" + }, + "id": "902909871aff0db" + }, + { + "cell_type": "markdown", + "source": [ + "## Copyrights:\n", + "Copyright 2023 Sony Semiconductor Israel, Inc. All rights reserved.\n", + "Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0\n", + "Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License." + ], + "metadata": { + "collapsed": false, + "id": "b2a030eb3ee565ef" + }, + "id": "b2a030eb3ee565ef" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 2 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython2", + "version": "2.7.6" + }, + "colab": { + "provenance": [] + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file