Relevance Forward Propagation (RFP)

This repository contains the code for the paper Efficient Data Source Relevance Quantification for Multi-Source Neural Networks (LINK).

The work proposes a Forward Relevance Propation approach to compute source-wise relevevances values within the neural network forward pass.

Prerequisites and Setup

This repository has been tested under Python 3.9.12 in a unix development environment.
For a setup, clone the repository and cdto the root of it.
Create a new environment and activate it, for instance, on unix via

python -m venv venv && source venv/bin/activate

Then install the needed packages via the requirements.txt file:

pip install --upgrade pip
pip install -r requirements.txt

Further, for working with the SEN12MS data set, the configurations for the data set have to be set in config/config.py.

Content

The folder code contains Jupyter Notebooks to reproduce the experiments and to explore the approach on the Multi-Source MNIST data set.

• MNIST Playground
  The notebook code/01_MNISTPlayGround.ipynb provides a pipeline to explore the different Multi-Source MNSIT setups, train networks and evaluate the relevance values.

• Regression Example
  The notebook code/02_RegressionExample.ipynb provides an example of the Forward Relevance Propagation applied to a regression task.

• Evaluation MNIST
  The notebook code/11_Evaluate_MNIST.ipynb provides a pipeline for the evaluation of the MNIST experiments presented in the paper.

• Evaluation SEN12MS
  The notebook code/12_Evaluate_SEN12MS.ipynb provides a pipeline to apply Relevance Forward Propagation to pre-trained networks provided by the authors of the data set.
  The pretrained networks can be found here.
  The original (clear) can be found here.
  The cloudy Version can be found here and here.

To use the provided scripts, the config file code/config/config.py has to be adjusted by setting the following variables:

• ckpt_path_resnet50_sen12ms_cfg: path to the pre-trained network here
• data_dir_sen12ms_cfg: path to the root folder of SEN12MS
• data_dir_sen12mscr_cfg: path to the root folder of SEN12MSCR
• label_split_dir_cfg folder containing the sample split fiels (see here)

• Evaluation of ADVANCE
  For the advance data, the original authors repository can simply be adapted by applying adjusting the framework provided by the authors. For the basic case, the script has to be adjusted to only consider the MSE, loss and dropping the Scene loss. The RFP reperensetation can be applied straight forward by using the functionalities in this repo (code/input_mapper.py). LINK

Citation

BibTeX format:

@article{name2024efficient,
  title={Efficient Data Source Relevance Quantification for Multi-Source Neural Networks},
  author={Gawlikowski, Jakob and Gottschling, Nina Maria},
  journal={British Machine Vision Conference 2024},
  year={2024}
}