Fine-tuning data requirement

[RKhobrag]

I have 2 questions regarding fine-tuning,

1. To fine-tune model like OpenLlama 7B, what is a decent size of training corpus needed? And does the data have to be in the instruction-style format or can it be plain text corpus?

2. The fine-tuning trains the model using what task exactly?

Thank you for all the help, and for taking this initiative!

[nmccoyhg]

I think having an example like @RKhobrag mentions would really give insight and interest into EasyLM and the OpenLLaMA initiatives you are undertaking. 👍