Security & Privacy in AIGC

A list of research towards security & privacy in AI-Generated Content.
Sorted by the appearance on arXiv.

Survey

• Security and Privacy on Generative Data in AIGC: A Survey
  Tao Wang, Yushu Zhang, Shuren Qi, Ruoyu Zhao, Zhihua Xia, Jian Weng

• On the Trustworthiness Landscape of State-of-the-art Generative Models: A Comprehensive Survey
  Mingyuan Fan, Cen Chen, Chengyu Wang, Jun Huang

Perturbations for Image Protection

• arXiv:2404.15081 Perturbing Attention Gives You More Bang for the Buck: Subtle Imaging Perturbations That Efficiently Fool Customized Diffusion Models
  Jingyao Xu, Yuetong Lu, Yandong Li, Siyang Lu, Dongdong Wang, Xiang Wei

• arXiv:2310.19248 IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AI
  Bochuan Cao, Changjiang Li, Ting Wang, Jinyuan Jia, Bo Li, Jinghui Chen

• arXiv:2309.11575 Distilling Adversarial Prompts from Safety Benchmarks: Report for the Adversarial Nibbler Challenge
  Manuel Brack, Patrick Schramowski, Kristian Kersting

• arXiv:2308.14761 Unified Concept Editing in Diffusion Models
  Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzyńska, David Bau

• arXiv:2308.10718 Backdooring Textual Inversion for Concept Censorship
  Yutong Wu, Jie Zhang, Florian Kerschbaum, Tianwei Zhang

• arXiv:2308.01937 Training Data Protection with Compositional Diffusion Models
  Aditya Golatkar, Alessandro Achille, Ashwin Swaminathan, Stefano Soatto

• arXiv:2308.04448 Dual Governance: The intersection of centralized regulation and crowdsourced safety mechanisms for Generative AI
  Avijit Ghosh, Dhanya Lakshmi

• arXiv:2307.16680 On the Trustworthiness Landscape of State-of-the-art Generative Models: A Comprehensive Survey
  Mingyuan Fan, Cen Chen, Chengyu Wang, Jun Huang

• arXiv:2307.13527 Not with my name! Inferring artists' names of input strings employed by Diffusion Models
  Roberto Leotta, Oliver Giudice, Luca Guarnera, Sebastiano Battiato

• arXiv:2307.12872 Data-free Black-box Attack based on Diffusion Model
  Mingwen Shao, Lingzhuang Meng, Yuanjian Qiao, Lixu Zhang, Wangmeng Zuo

• arXiv:2307.03108 How to Detect Unauthorized Data Usages in Text-to-image Diffusion Models
  Zhenting Wang, Chen Chen, Yuchen Liu, Lingjuan Lyu, Dimitris Metaxas, Shiqing Ma

• arXiv:2306.15774 Next Steps for Human-Centered Generative AI: A Technical Perspective
  Xiang 'Anthony' Chen, Jeff Burke, Ruofei Du, Matthew K. Hong, Jennifer Jacobs, Philippe Laban, Dingzeyu Li, Nanyun Peng, Karl D. D. Willis, Chien-Sheng Wu, Bolei Zhou

• arXiv:2306.09776 Inspire Creativity with ORIBA: Transform Artists' Original Characters into Chatbots through Large Language Model
  Yuqian Sun, Xingyu Li, Ze Gao

• arXiv:2306.08310 TWIGMA: A dataset of AI-Generated Images with Metadata From Twitter
  Yiqun Chen, James Zou

• arXiv:2306.08257 On the Robustness of Latent Diffusion Models
  Jianping Zhang, Zhuoer Xu, Shiwen Cui, Changhua Meng, Weibin Wu, Michael R. Lyu

• arXiv:2306.07754 Generative Watermarking Against Unauthorized Subject-Driven Image Synthesis
  Yihan Ma, Zhengyu Zhao, Xinlei He, Zheng Li, Michael Backes, Yang Zhang

• arXiv:2306.05949 Evaluating the Social Impact of Generative AI Systems in Systems and Society
  Irene Solaiman, Zeerak Talat, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Hal Daumé III, Jesse Dodge, Ellie Evans, Sara Hooker, Yacine Jernite, Alexandra Sasha Luccioni, Alberto Lusoli, Margaret Mitchell, Jessica Newman, Marie-Therese Png, Andrew Strait, Apostol Vassilev

• arXiv:2306.04141 Art and the science of generative AI: A deeper dive
  Ziv Epstein, Aaron Hertzmann, Laura Herman, Robert Mahari, Morgan R. Frank, Matthew Groh, Hope Schroeder, Amy Smith, Memo Akten, Jessica Fjeld, Hany Farid, Neil Leach, Alex Pentland, Olga Russakovsky

• arXiv:2306.01902 Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation
  Zhengyue Zhao, Jinhao Duan, Xing Hu, Kaidi Xu, Chenan Wang, Rui Zhang, Zidong Du, Qi Guo, Yunji Chen

• arXiv:2306.00080 AI Imagery and the Overton Window
  Sarah K. Amer

• arXiv:2306.00419 Challenges and Remedies to Privacy and Security in AIGC: Exploring the Potential of Privacy Computing, Blockchain, and Beyond
  Chuan Chen, Zhenpeng Wu, Yanyi Lai, Wenlin Ou, Tianchi Liao, Zibin Zheng

• arXiv:2305.18615 Stronger Together: on the Articulation of Ethical Charters, Legal Tools, and Technical Documentation in ML
  Giada Pistilli, Carlos Munoz Ferrandis, Yacine Jernite, Margaret Mitchell

• arXiv:2305.16934 On Evaluating Adversarial Robustness of Large Vision-Language Models
  Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, Min Lin

• arXiv:2305.13238 The Dimensions of Data Labor: A Road Map for Researchers, Activists, and Policymakers to Empower Data Producers
  Hanlin Li, Nicholas Vincent, Stevie Chancellor, Brent Hecht

• arXiv:2305.12683 Mist: Towards Improved Adversarial Examples for Diffusion Models
  Chumeng Liang, Xiaoyu Wu

• arXiv:2305.12502 Watermarking Diffusion Model
  Yugeng Liu, Zheng Li, Michael Backes, Yun Shen, Yang Zhang

• arXiv:2305.12015 Inventing painting styles through natural inspiration
  Nilin Abrahamsen, Jiahao Yao

• arXiv:2304.03545 AI Model Disgorgement: Methods and Choices
  Alessandro Achille, Michael Kearns, Carson Klingenberg, Stefano Soatto

• arXiv:2304.02234 JPEG Compressed Images Can Bypass Protections Against AI Editing
  Pedro Sandoval-Segura, Jonas Geiping, Tom Goldstein

• arXiv:2303.16378 A Pilot Study of Query-Free Adversarial Attack against Stable Diffusion
  Haomin Zhuang, Yihua Zhang, Sijia Liu

• arXiv:2303.15433 Anti-DreamBooth: Protecting users from personalized text-to-image synthesis
  Thanh Van Le, Hao Phung, Thuan Hoang Nguyen, Quan Dao, Ngoc Tran, Anh Tran

• arXiv:2303.13516 Ablating Concepts in Text-to-Image Diffusion Models
  Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, Jun-Yan Zhu

• arXiv:2303.07345 Erasing Concepts from Diffusion Models
  Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufman, David Bau

• arXiv:2302.06588 Raising the Cost of Malicious AI-Powered Image Editing. ICML 2023
  Hadi Salman, Alaa Khaddaj, Guillaume Leclerc, Andrew Ilyas, Aleksander Madry

• arXiv:2302.04578 Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examples. ICML 2023
  Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yiming Xue, Tao Song, Zhengui Xue, Ruhui Ma, Haibing Guan

• arXiv:2302.04222 GLAZE: Protecting Artists from Style Mimicry by Text-to-Image Models. USENIX Security 2023
  Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, Ben Y. Zhao

• arXiv:2212.03860 Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models
  Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, Tom Goldstein

Watermark

• arXiv:2405.02365 Adaptive and robust watermark against model extraction attack
  Kaiyi Pang, Tao Qi, Chuhan Wu, Minhao Bai

• arXiv:2311.12832 Toward effective protection against diffusion-based mimicry through score distillation ICLR2024
  Haotian Xue, Chumeng Liang, Xiaoyu Wu, Yongxin Chen

• arXiv:2404.13518 Reliable Model Watermarking: Defending Against Theft without Compromising on Evasion
  Hongyu Zhu, Sichu Liang, Wentao Hu, Fangqi Li, Ju Jia, Shilin Wang

• arXiv:2404.04956 Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models
  Zijin Yang, Kai Zeng, Kejiang Chen, Han Fang, Weiming Zhang, Nenghai Yu

• arXiv:2403.10893 A Watermark-Conditioned Diffusion Model for IP Protection
  Rui Min, Sen Li, Hongyang Chen, Minhao Cheng

• arXiv:2401.08573 Benchmarking the Robustness of Image Watermarks
  Bang An, Mucong Ding, Tahseen Rabbani, Aakriti Agrawal, Yuancheng Xu, Chenghao Deng, Sicheng Zhu, Abdirisak Mohamed, Yuxin Wen, Tom Goldstein, Furong Huang

• arXiv:2312.08883 EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protection CVPR 2024
  Xuanyu Zhang, Runyi Li, Jiwen Yu, Youmin Xu, Weiqi Li, Jian Zhang

• arXiv:2311.13713 A Somewhat Robust Image Watermark against Diffusion-based Editing Models
  Mingtian Tan, Tianhao Wang, Somesh Jha

• arXiv:2310.07726 Warfare:Breaking the Watermark Protection of AI-Generated Content
  Guanlin Li, Yifei Chen, Jie Zhang, Jiwei Li, Shangwei Guo, Tianwei Zhang

• arXiv:2309.16952 Leveraging Optimization for Adaptive Attacks on Image Watermarks ICLR 2024
  Nils Lukas, Abdulrahman Diaa, Lucas Fenaux, Florian Kerschbaum

• arXiv:2309.05940 Catch You Everything Everywhere: Guarding Textual Inversion via Concept Watermarking
  Weitao Feng, Jiyan He, Jie Zhang, Tianwei Zhang, Wenbo Zhou, Weiming Zhang, Nenghai Yu

• arXiv:2306.03436 Intellectual Property Protection of Diffusion Models via the Watermark Diffusion Process
  Sen Peng, Yufei Chen, Cong Wang, Xiaohua Jia

• arXiv:2306.04642 DiffusionShield: A Watermark for Copyright Protection against Generative Diffusion Models
  Yingqian Cui, Jie Ren, Han Xu, Pengfei He, Hui Liu, Lichao Sun, Yue Xing, Jiliang Tang

• arXiv:2306.01953 Invisible Image Watermarks Are Provably Removable Using Generative AI
  Xuandong Zhao, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, Lei Li

• arXiv:2305.20030 Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust NeurIPS 2023
  Yuxin Wen, John Kirchenbauer, Jonas Geiping, Tom Goldstein

• arXiv:2305.03807 Evading Watermark based Detection of AI-Generated Content
  Zhengyuan Jiang, Jinghuai Zhang, Neil Zhenqiang Gong

• arXiv:2303.15435 The Stable Signature: Rooting Watermarks in Latent Diffusion Models ICCV 2023
  Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, Teddy Furon

• arXiv:2303.10137 A Recipe for Watermarking Diffusion Models
  Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Ngai-Man Cheung, Min Lin

• arXiv:2209.03466 Supervised GAN Watermarking for Intellectual Property Protection
  Jianwei Fei, Zhihua Xia, Benedetta Tondi, Mauro Barni

• arXiv:2102.04362 Protecting Intellectual Property of Generative Adversarial Networks from Ambiguity Attack CVPR 2021
  Ding Sheng Ong, Chee Seng Chan, Kam Woh Ng, Lixin Fan, Qiang Yang

• arXiv:2007.08457 Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data ICCV 2021
  Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, Mario Fritz

Membership Inference Attack

• arXiv:2312.08207 Black-box Membership Inference Attacks against Fine-tuned Diffusion Models
  Yan Pang, Tianhao Wang

• arXiv:2312.05140 Membership Inference Attacks on Diffusion Models via Quantile Regression
  Shuai Tang, Zhiwei Steven Wu, Sergul Aydore, Michael Kearns, Aaron Roth

• arXiv:2308.12143 A Probabilistic Fluctuation based Membership Inference Attack for Diffusion Models
  Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, Tao Jiang

• arXiv:2308.06405 White-box Membership Inference Attacks against Diffusion Models
  Yan Pang, Tianhao Wang, Xuhui Kang, Mengdi Huai, Yang Zhang

• arXiv:2306.12983 Towards More Realistic Membership Inference Attacks on Large Diffusion Models
  Jan Dubiński, Antoni Kowalczuk, Stanisław Pawlak, Przemysław Rokita, Tomasz Trzciński, Paweł Morawiecki

• arXiv:2305.18355 An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization
  Fei Kong, Jinhao Duan, RuiPeng Ma, Hengtao Shen, Xiaofeng Zhu, Xiaoshuang Shi, Kaidi Xu

• arXiv:2305.08694 A Reproducible Extraction of Training Images from Diffusion Models
  Ryan Webster

• arXiv:2302.07801 Data Forensics in Diffusion Models: A Systematic Analysis of Membership Privacy
  Derui Zhu, Dingfan Chen, Jens Grossklags, Mario Fritz

• arXiv:2302.03262 Membership Inference Attacks against Diffusion Models
  Tomoya Matsumoto, Takayuki Miura, Naoto Yanai

• arXiv:2302.01316 Are Diffusion Models Vulnerable to Membership Inference Attacks? Jinhao Duan, Fei Kong, Shiqi Wang, Xiaoshuang Shi, Kaidi Xu

• arXiv:2301.13188 Extracting Training Data from Diffusion Models
  Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, Eric Wallace

• arXiv:2301.09956 Membership Inference of Diffusion Models
  Hailong Hu, Jun Pang

• arXiv:2210.00968 Membership Inference Attacks Against Text-to-image Generation Models
  Yixin Wu, Ning Yu, Zheng Li, Michael Backes, Yang Zhang

Multimodal Foundation Modal

• arXiv:2402.08577 Test-Time Backdoor Attacks on Multimodal Large Language Models
  Dong Lu, Tianyu Pang, Chao Du, Qian Liu, Xianjun Yang, Min Lin

• arXiv:2309.11751 How Robust is Google's Bard to Adversarial Image Attacks?
  Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, Jun Zhu

• arXiv:2308.10741 On the Adversarial Robustness of Multi-Modal Foundation Models
  Christian Schlarmann, Matthias Hein

• arXiv:2306.13213 Visual Adversarial Examples Jailbreak Aligned Large Language Models
  Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, Prateek Mittal

• arXiv:2305.16934 On Evaluating Adversarial Robustness of Large Vision-Language Models
  Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, Min Lin