DPO Fine-Tuning

[AnirudhJM24]

The repository contains examples to fine-tune the model using Supervised Fine Tuning. I wish to add examples of Transformer Reinforcement Learning (TRL) particulary Direct Policy Optimization (DPO)

[ariG23498]

Hey @AnirudhJM24

I really like the idea, but would also ask you to share a rough colab notebook for this. I don't want a very complicated setup for SFT in the repository. Having said that, if you can showcase the workflow in a very simple way, I would be open to adding it.

Also do take a look at the /fine_tune directory for inspiration.