SAILS — Surrogate-based Analysis of Interactions via Local Effect Smooths

This repository contains the code for the paper SAILS: Surrogate-based Analysis of Interactions via Local Effect Smooths. SAILS is a model-agnostic framework for analyzing pairwise feature interactions in black-box models. It fits interpretable GAM surrogates to ALE local effects per interval of a feature of interest, enabling interaction detection, categorization (linear, product-separable, or non-separable), and tailored visualization.

Installation & Set up

1.) Clone or fork this repository

2.) Install pipx:

pip install --user pipx

3.) Install Poetry:

pipx install poetry

3a.) Optionally (if you want the env-folder to be created in your project):

poetry config virtualenvs.in-project true

4.) Install this project:

poetry install

4a.) If the specified Python version (3.11 for this project) is not found:

If the required Python version is not installed on your device, install the corresponding Python version from the official python.org website.

Then run

poetry env use <path-to-your-python-version>

and install the project:

poetry install

Dependencies

Exact dependencies are documented in the poetry.lock. Executing poetry install will produce an environment with exactly the dependencies used for the experiments and application.

Repository Contents

interaction_effects/          # Core Python package
  ale_gam.py                  # GAM surrogate fitting on ALE local effects (main method)
  explainer.py                # High-level SAILS explainer class
  feature_effects.py          # ALE local effect estimation
  plotting.py                 # Visualization utilities
  data_generation.py          # Simulation data generation
  groundtruths.py             # Ground truth interaction functions for evaluation
  regional_effects.py         # Regional effects (GADGET/REPID) baseline
  helpers/                    # Internal utilities (GAM, general)
  experiments/                # Experiment pipeline
    main.py                   # Entry point (run via `poetry run run_experiment`)
    run_experiment.py         # Experiment orchestration
    model_training.py         # ML model training and hyperparameter tuning
    model_eval.py             # Model evaluation
    saving.py                 # Result persistence
    logging.py                # Experiment logging
    configs/                  # Experiment and base configurations
    evaluation/               # Result loading, plotting, and table export

notebooks/                    # Analysis notebooks
  motivational_example.ipynb          # Fig. 1 — motivating example
  general_idea_example.ipynb          # included in Fig. 2 — general idea illustration
  intuition_linearity_and_ratio.ipynb # Fig. 3 — linearity/ratio intuition
  model_evaluation.ipynb              # Tab. A.1 — ML model performance
  simulation_eval_significance.ipynb  # Fig. 4, A.2 — interaction detection results
  simulation_eval_categorization.ipynb# Fig. 5, A.3–A.4 — categorization results
  simulation_eval_visualization.ipynb # Fig. 6–7, A.1, A.5–A.11 — visualization results
  simulation_eval_surrogate_fit.ipynb # Tab. A.2 — surrogate goodness-of-fit
  real_world_application.ipynb        # Fig. 8, Tab. 2 — Tetouan application

results/                      # Experiment outputs (generated by run_experiment)
  <experiment>/
    results/                  # SQLite databases with interaction analysis results
    explainer/                # Fitted SAILS explainer objects (.joblib)
    tuning_studies/           # Optuna hyperparameter tuning databases
    logs/                     # Experiment log and config snapshot

Reproduce the Experiments

All our simulation experiments are fully reproducible. To reproduce the simulation results, simply run the following command (after having completed "Installation & Set up"):

poetry run run_experiment --experiment twoway_indep

This runs the twoway_indep experiment as specified in the EXPERIMENT_DICT. The following experiments are available:

Setting	Specifier
I.	"twoway_indep"
II.	"twoway_corr"
III.	"higher_order_simple"
IV.	"higher_order_complex"

The results are saved in a directory named after the experiment specifier in a results directory. The logging output is saved in a logs subdirectory within that experiment directory. Note that if the experiment directory already exists, the run will fail (as safety measure to avoid overwriting existing results).

Inspect Experiment Results and Reproduce Figures and Tables

After running all four experiments, open the corresponding notebook in notebooks/ with the virtual environment induced by poetry as kernel and run all cells. The notebooks read results directly from the results/ directory.

Notebook	Paper artifacts
simulation_eval_significance.ipynb	Fig. 4, Fig. A.2
simulation_eval_categorization.ipynb	Fig. 5, Fig. A.3–A.4
simulation_eval_visualization.ipynb	Fig. 6–7, Fig. A.1, Fig. A.5–A.11
simulation_eval_surrogate_fit.ipynb	Tab. A.2
model_evaluation.ipynb	Tab. A.1

The following notebooks are self-contained and do not depend on pre-run experiment results:

Notebook	Paper artifacts
motivational_example.ipynb	Fig. 1
general_idea_example.ipynb	Fig. 2 (only plot in step 1)
intuition_linearity_and_ratio.ipynb	Fig. 3

Reproduce the Application

The application notebook (notebooks/real_world_application.ipynb) is self-contained. The Tetouan city power consumption dataset is fetched automatically from the UCI ML Repository via ucimlrepo:

from ucimlrepo import fetch_ucirepo
dataset = fetch_ucirepo(id=849)

Open the notebook with the virtual environment induced by poetry as kernel and run all cells to reproduce Tab. 2 and Fig. 8.

Apply our Method

SAILS can be applied to any scikit-learn-compatible model and a pandas DataFrame of training data.

from interaction_effects.explainer import SailsExplainer

# model: any fitted scikit-learn-compatible model
# X_train: pd.DataFrame of training features
# feature_of_interest: name of the column to analyze interactions for
explainer = SailsExplainer(model=model, X=X_train, feature_of_interest="x_j")
explainer.fit()

# (i) Detect interacting features
explainer.compute_interaction_significance()
print(explainer.global_interaction_significance)

# (ii) Categorize interactions (linear / product-separable / non-separable)
explainer.compute_interaction_categorization()
print(explainer.interaction_categorization)

# (iii) Visualize interactions (integrated smooth terms)
explainer.compute_integrated_splines()

import pandas as pd
from interaction_effects.plotting import plot_grid_lines_interval_labels, plot_interaction_curves

spline_data = explainer.local_effect_splines_integrated
interval_labels = pd.DataFrame({
    "FOI.value": explainer.local_effects.interval_centers,
    "FOI.interval": explainer.local_effects.intervals,
})
spline_data = spline_data.merge(interval_labels, on="FOI.value", how="left")
spline_data["FOI.interval"] = pd.Categorical(
    spline_data["FOI.interval"],
    categories=explainer.local_effects.intervals,
    ordered=True,
)
spline_data = spline_data.sort_values(["interacting_feature.name", "FOI.interval"])

plot_interaction_curves(
    spline_data,
    plot_func=plot_grid_lines_interval_labels,
    id_var="interacting_feature.value",
    value_var="effect",
    group_var="interacting_feature.name",
    legend_title=f"Intervals of {explainer.feature_of_interest}",
    sharey=False,
)

See notebooks/real_world_application.ipynb for a full worked example including visualization.