Official implementation of "SS-ADA: A Semi-Supervised Active Domain Adaptation Framework for Semantic Segmentation". Submitted to T-ITS on May **, 2024.
Semantic segmentation plays an important role in understanding the environment but the labeling budget is expensive and time-consuming. Semi-supervised semantic segmentation methods have been proposed to leverage large quantities of unlabeled images to reduce annotation costs. Despite this, their performance still falls short of fully-supervised learning. A significant shortcoming is that they randomly select unlabeled images for human annotation, neglecting the assessment of sample value for model training. Active domain adaptation methods aim to transfer knowledge from a labeled source domain to an unseen target domain with a few yet valuable samples on target domain. However, existing active domain adaptation methods primarily rely on pixel or region-level acquisition strategies, which are not feasible in practical scenarios. This paper proposes a novel semi-supervised active domain adaptation framework named SS-ADA for semantic segmentation. SS-ADA integrates active learning into semi-supervised semantic segmentation to achieve the accuracy of fully-supervised learning with a limited amount of labeled data from the target domain. Image-level acquisition strategy Entropy-IoU Sum is developed to use both pixel and class-level information for sample selection. Additionally, we design an IoU-based class weighting strategy to alleviate the class imbalance problem using annotations from active learning. We conduct extensive experiments on synthetic-to-real and real-to-real domain adaptation settings. The results demonstrate the effectiveness of our method. SS-ADA can achieve or even surpass the accuracy of its fully-supervised learning counterpart with only 25% of the target labeled data when using a real-time segmentation model BiSeNet. The code for SS-ADA is available at \url{https://github.com/ywher/SS-ADA}
I verified this reporitory in Ubuntu20.04 with Anaconda3, Pytorch 1.12.0, CUDA 11.3, 3090 GPU.
First, create the environment named ss-ada and activate the ss-ada environment through:
conda create -n ss-ada python=3.8 -y
conda activate ss-ada
Then install the required packages though:
pip install torch==1.12.0+cu113 torchvision==0.13.0+cu113 torchaudio==0.12.0 --extra-index-url https://download.pytorch.org/whl/cu113
pip install -r requirements.txt
Download the code from github and change the directory:
git clone https://github.com/ywher/SS-ADA
cd SS-ADA
Here we only show how to set the GTA5-to-Cityscapes and bev2023-to-bev2024 settings. You can prepare the SYNTHIA-to-Cityscapes, Cityscapes-to-ACDC , and Cityscapes-to-FisheyeCampus datasets similarly.
Download bev2023, bev2024, GTA5, and Cityscapes datasets then organize the folder as follows:
|SS-ADA/data
│ ├── bev2023/
| | ├── image/
| | | ├── train/
| | | ├── val/
| | ├── label/
| | | ├── train/
| | | ├── val/
│ ├── bev2024/
| | ├── image/
| | | ├── train/
| | | ├── val/
| | ├── label/
| | | ├── train/
| | | ├── val/
│ ├── cityscapes/
| | ├── leftImg8bit/
| | | ├── train/
| | | ├── val/
| | | ├── test/
| | ├── gtFine/
| | | ├── train/
| | | ├── val/
│ ├── gtav/
| | ├── images/
| | ├── labels/
...
Downlaod the ImageNet pretrained ResNet18 and put it in /pretrained folder
https://download.pytorch.org/models/resnet18-5c106cde.pth
We use one 3090 GPU for training and evaluation.
Remember to change the work root from "/media/ywh/pool1/yanweihao/projects/active_learning/SS-ADA" in configs/*.yaml to your own SS-ADA root
TBD
set the scripts/train_bisenet.sh as following:
dataset='bev_2023'
method='supervised_bisenetv1_tar'
exp='bisenetv1'
split='110'
config_name='parking_bev2023_bisenetv1'
Then run the training bash, (n_gpus=2, port=1008)
bash scripts/train_bisenetv1.sh 2 10008
set the use source in configs/parking_bev2024_bisenetv1.yaml to False
source:
use_source: False
set the scripts/train_bisenet.sh as following:
dataset='bev_2024'
method='supervised_bisenetv1_tar'
exp='bisenetv1'
split='140'
config_name='parking_bev2024_bisenetv1'
Then run the training bash, (n_gpus=2, port=1008)
bash scripts/train_bisenetv1.sh 2 10008
set the use source in configs/parking_bev2024_bisenetv1.yaml to True
source:
use_source: True
type: bev_2023
data_root: /your_path_to_SS-ADA/SS-ADA/data/bev_2023
data_list: /your_path_to_SS-ADA/SS-ADA/data/train_list/bev_2023_train_list.txt
set the scripts/train_bisenet.sh as following:
dataset='bev_2024'
method='supervised_bisenetv1_both'
exp='bisenetv1'
split='140'
config_name='parking_bev2024_bisenetv1'
Then run the training bash, (n_gpus=2, port=1008)
bash scripts/train_bisenetv1.sh 2 10008
set the use source in configs/parking_bev2024_acda_bisenetv1_single,yaml to True
and set the n_sup and active.total ratio to the corresponding setting.
Here we take 50% of target labeled data,n_sup=70 and total_ratio=0.50 as example. (70, 0.5; 35, 0.25...))
n_sup: 70
source:
use_source: True
active:
init_ratio: 0.01
total_ratio: 0.50
set the scripts/train_acda_bisenet_single.sh as following:
dataset='bev_2024'
method='ss_ada_bisenetv1_single'
exp='bisenetv1'
split='70'
config_name='parking_bev2024_acda_bisenetv1_single'
init_split=1
Then run the training bash, (n_gpus=1)
bash scripts/train_acda_bisenetv1_single
TBD
TBD
TBD
Our trained models can be found in the following links:
- 50% of the target labeled data: https://drive.google.com/file/d/10gki3Fk5V0ZWwiWYZ80DLY-aPBJdD9pk/view?usp=drive_link
- 25% of the target labeled data: https://drive.google.com/file/d/1ApSkCi7ELP71oEvcvqi9a0FsjGV6qKyQ/view?usp=drive_link
- 12.5% of the target labeled data: https://drive.google.com/file/d/1YThOOARCUUrAsGxyt11hv_KfGtTVKOY0/view?usp=drive_link
change the val dataset from bev2023 to bev2024 in parking_bev2023_bisenetv1.yaml
val:
dataset: bev_2024
data_root: /media/ywh/pool1/yanweihao/projects/active_learning/SS-ADA/data/bev_2024
set in scripts/eval_bisenetv1.sh and run "bash scripts/eval_bisenetv1.sh"
dataset="bev_2023"
config_path="configs/parking_bev2023_bisenetv1.yaml"
eval_mode="original"
exp_folder="supervised_bisenetv1_tar"
split=110
123
set in scripts/eval_bisenetv1.sh and run "bash scripts/eval_bisenetv1.sh"
dataset="bev_2024"
config_path="configs/parking_bev2024_bisenetv1.yaml"
eval_mode="original"
exp_folder="supervised_bisenetv1_tar"
split=140
set in scripts/eval_bisenetv1.sh and run "bash scripts/eval_bisenetv1.sh"
dataset="bev_2024"
config_path="configs/parking_bev2024_bisenetv1.yaml"
eval_mode="original"
exp_folder="supervised_bisenetv1_both"
split=140
Our trained models can be found in the following links:
- 50% of the target labeled data: https://drive.google.com/drive/folders/1qVgo4uKCH9ZqVFbwAdJ09yLDLQ9wnG3v?usp=sharing
- 25% of the target labeled data:https://drive.google.com/drive/folders/1NS2da6Dj3thafgNm5MqakZgJO6_FdlWr?usp=sharing
The evaluation results of our trained models on bev2024 validation set are listed in the following.
| Setting | best.pth (mIoU %) | latest.pth (mIoU %) |
|---|---|---|
| bev2023 sup only | 38.09 | 38.09 |
| bev2024 supervised learning | 71.63 | 70.20 |
| bev2024 joint training | 74.41 | 72.79 |
| ss-ada 50% data | 73.09 | 72.83 |
| ss-ada 25% data | 71.67 | 71.06 |
Some of the code is borrowed from Unimatch and BiSeNet
Thanks a lot for their great work!
If you use this code in your research please consider citing our work
TBD
Weihao Yan: [email protected]