MODEL_ZOO.md

Panoptic-DeepLab Model Zoo and Baselines

Introduction

This file documents a large collection of baselines for Panoptic-DeepLab. We are planning to provide results Panoptic-DeepLab with different backbones as well as projects that are built on Panoptic-DeepLab.

Currently we only have results on the Cityscapes panoptic segmentation benchmark, results on other datasets like COCO and Mapillary Vistas are still under development.

You are wellcome to put results of your model in this model zoo if it is based on Panoptic-DeepLab (e.g. new backbone, new head design, etc).

Cityscapes baselines

ResNet

By default, models are trained with a batch size of 8 with 8 GPUs. You can also train it with batchsize of 4 using 4 GPUs (without changing learning rate and total number of iterations).

We use the TorchVision ResNet implementation, which is not exactly the same as the TensorFlow ResNet implementation.

For ResNet-50 model, 11G memory should be enough. You will need larger memory for models larger than R50.

Name	PQ	SQ	RQ	AP	mIoU	Model
R50-os32	59.8	80.0	73.5	26.9 / 28.3	78.6	Google Drive
R101-os32	60.3	80.8	73.6	27.7 / 30.2	78.4	Google Drive
X101-32x8d-os32	61.4	80.8	74.9	28.9 / 30.3	79.6	Google Drive

Note:

• R50/R101: ResNet-50 and ResNet-101
• X101: ResNext-101
• 2 AP numbers refer to different ways to calculate condidence score: semantic / semantic x instance

Xception models

Name	PQ	SQ	RQ	AP	mIoU	Model
X65-os32	61.8	81.1	75.2	30.9 / 31.7	79.6	Google Drive

Note:

• X65: Xception-65
• 2 AP numbers refer to different ways to calculate condidence score: semantic / semantic x instance

Mobile models

Name	PQ	SQ	RQ	AP	mIoU	Model
MBNV2-1.0-os32	55.1	79.4	68.0	20.6 / 23.3	75.8	Google Drive

Note:

• MBNV2-1.0: MobileNetV2 with width multiplier 1.0
• 2 AP numbers refer to different ways to calculate condidence score: semantic / semantic x instance

HRNet series (Under Progress)

We trained HRNet-W48 baseline on Cityscapes with 8 Tesla V100 GPUs, with a batchsize 8. We use the HRNet-Semantic-Segmentation HRNet implementation. All the results are under progress and we are making efforts to finetune various hyperparameters to achieve better performance.

We use Pytorch1.5 in all of our experiments. The docker is rainbowsecret/pytorch1.5:latest.

Name	PQ	SQ	RQ	AP	mIoU	Model
H48-os4	63.4	81.5	76.7	29.9 / 29.6	80.9	Google Drive

Note:

• H48: HRNet with width 48
• 2 AP numbers refer to different ways to calculate condidence score: semantic / semantic x instance