DP-Sniper

A machine-learning-based tool for discovering differential privacy violations in black-box algorithms.

Install

We recommend installing DP-Sniper using conda.

After installing conda, you can install DP-Sniper by running its installation script install.sh:

bash ./install.sh

You can ignore the warning ResourceWarning: unclosed running multiprocessing pool.

Note: The above steps are sufficient to use the main package dpsniper. If you would like to run the experiments from the SP 2021 research paper, you have to follow additional installation steps as described in eval_sp2021/README.md).

Basic Usage

The following command tests the differential privacy of the Laplace mechanism, explained in detail in file dpsniper/example.py:

conda activate dp-sniper
python dpsniper/example.py # may take a while due to an extensive final confirmation

This commands stores temporary outputs and log files to the folder example_outputs of the current working directory.

Testing Your Own Mechanism

DP-Sniper is a black-box approach. To run DP-Sniper or DD-Search on your own mechanism, you only have to implement the method m of the abstract class Mechanism defined in dpsniper/mechanisms/abstract.py and modify the code snippet in dpsniper/example.py. See dpsniper/mechanisms for example implementations of popular mechanisms.

Publication

This is an implementation of the approach presented in the following research paper:

B. Bichsel, S. Steffen, I. Bogunovic and M. Vechev. 2021. DP-Sniper: Black-Box Discovery of Differential Privacy Violations using Classifiers. In IEEE Symposium on Security and Privacy (SP 2021).

The main algorithms DD-Search and DP-Sniper from the paper can be found in dpsniper/search/ddsearch.py and dpsniper/attack/dpsniper.py, respectively.

Citing this Work

You are encouraged to cite the above publication using the following BibTeX entry if you use DP-Sniper for academic research.

@inproceedings{bichsel2021dpsniper,
    author={Bichsel, Benjamin and Steffen, Samuel and Bogunovic, Ilija and Vechev, Martin},
    title = {DP-Sniper: Black-Box Discovery of Differential Privacy Violations using Classifiers},
    booktitle = {2021 IEEE Symposium on Security and Privacy (SP)},
    year = {2021},
    pages = {391-409},
    doi = {10.1109/SP40001.2021.00081},
    url = {https://doi.org/10.1109/SP40001.2021.00081},
    publisher = {IEEE Computer Society},
    address = {Los Alamitos, CA, USA},
    month = {may}
}

Evaluation

You can find instructions on how to reproduce the evaluation results of our paper in the folder eval_sp2021.

License

MIT License, see LICENSE.

This repository includes third-party code from statdp, marked as MIT License, Copyright (c) 2018-2019 Yuxin Wang.