simCLR Implementation using Pytorch

In this work we attempt to implement the pipeline detailed in the simCLR_paper, in our implementation we are using ResNet-50 backbone and CIFAR10 training with ImageNet Initialization.

The project is splitted into two parts:

• Self-Supervised learning followed by finetuning for classification.
• Supervised learning using similar settings for benchmarking

Default Settings

• Optimizer Choice:
  1. LARS optimizer:
     • lr = 0.075 * sqrt(batch_size)
     • eps=1e-8
     • trust_coef=0.001
  2. Adam:
     • lr = 0.3 * batch_size/256
     • Fine-tuning : lr = 0.01

Run the project

In this project, we have two steps: training and predicting. In the predict step, you can upload any image from your laptop and predict it. Let's show you how to run the project.

If you do not have venv package, please refer to this link

Create virtual environment

$ python3 -m venv ENV_NAME

Activate your environment

$ source ENV_NAME/bin/activate

Requirement installations

To run this, make sure to install all the requirements by:

$ pip install -r requirements.txt 

Training the Supervised Model

$ python3 main.py --model resnet --num_epochs

Training the Self-Supervised Model with Fine tuning

$ python3 main.py --model simclr --num_epochs

Make prediction

$python3 predict.py --image_path "./data/Images/cat.0.jpg"

Related Papers

• Resnet
• A Simple Framework for Contrastive Learning of Visual Representations

Contributors

• Allassan
• Mohamed Elabass
• Fanta SOUMAHORO
• Amel Abdelraheem