Adding Support for Hubble Suite

[aflah02]

Tasks

• Benchmark
• Unlearning method
• Evaluation
• Dataset
• None of the above

Feature request

Hey
We're soon releasing Hubble, a testbed to study memorization in LLMs and we ran a bunch of our MIA experiments using Open-Unlearning (present on our fork here - https://github.com/aflah02/open-unlearning/tree/Hubble_MIAs). We would like to bring some of the things upstream (in particular the model configs as well as running the MIAs with our data. We'd love to hear if this is something you're interested in adding?

Motivation

To present a testbed for future researchers to study MIAs more closely and carefully

Implementation

Present here - https://github.com/aflah02/open-unlearning/tree/Hubble_MIAs

[aflah02]

Hubble is now public: https://allegro-lab.github.io/hubble/
@molereddy @pratyushmaini - Any thoughts in making it a direct offering for easy use?

[molereddy]

Congrats on what seems like a great work!

Please create a PR integrating this with OpenUnlearning. Try to ensure that it's well-tested, design it so the implementation is clean, and seamlessly lines up with the rest of OpenUnlearning to the best extent possible. We will help you merge it in!