SODIUM

This is the implementation for SODIUM (Semantic Ood Detector In Unseen doMains).

Requirements

Python: 3.9.13
PyTorch: 1.12.1

Quick Start

We use sweep command to do model selection. To test the sweep launch, you can run the following dummy test:

python -m domainbed.scripts.sweep launch\
       --data_dir=domainbed/data\
       --output_dir=domainbed/sweep_outputs/PACS/SODIUM/test\
       --command_launcher dummy\
       --algorithms SODIUM\
       --datasets PACS\

You can launch a sweep using the command:

python -m domainbed.scripts.sweep launch\
       --data_dir=domainbed/data\
       --output_dir=domainbed/sweep_outputs/PACS/SODIUM/ood_6\
       --command_launcher same_type_gpu\
       --algorithms SODIUM\
       --datasets PACS\
       --skip_confirmation\
       --n_trials 2\
       --n_hparams 20\
       --save_model_every_checkpoint\
       --ood_classes 6\
       --auto_rerun_incomplete\
       --first_gpu_id 0\
       --n_gpus 1\
       --jobs_per_gpu 1\
       > log_sweep.txt

To use a customized a command launcher, add it to "domainbed/command_launchers.py".

When all models are trained, you can run the following commands to get the domain generalization and OOD detection results:

python -m domainbed.scripts.collect_results\
       --input_dir=domainbed/sweep_outputs/PACS/SODIUM/ood_6\
       > log_dg.txt

python -m domainbed.scripts.ood_detection\
       --input_dir=domainbed/sweep_outputs/PACS/SODIUM/ood_6\
       --algorithm SODIUM\
       --dataset PACS\
       --ood_class 6\
       > log_ood.txt

Generate DG results summary for different OOD class selections:

python -m domainbed.scripts.dg_results_summary\
       --input_dir=domainbed/sweep_outputs/PACS/SODIUM\
       --prefix=ood\
       --ood_classes 3 6

Generate OOD detection results summary for different OOD class selections:

python -m domainbed.scripts.ood_results_summary\
       --input_dir=domainbed/sweep_outputs/PACS/SODIUM\
       --prefix=ood\
       --dataset PACS\
       --ood_classes 3 6

Delete saved checkpoints except for the best ones:

python -m domainbed.scripts.clean_saved_models\
     --input_dir=domainbed/sweep_outputs/PACS/SODIUM/ood_6\
     --algorithm SODIUM\
     --dataset PACS