The purpose of this exercise was to gain some general understanding of the process of fine-tuning a transformer model like BERT. I gained an understanding of how to use HuggingFace models and PyTorch, and got a handle on a more complex implementation of the classic sentiment analysis task. I wanted to incorporate some syntactic parsing into the implementation, so I used the SST-2 treebank with NLTK. I can't take full credit for this implementation, as I used an LLM to guide my work. Nonetheless, I think this was a great way to learn about some intimidating topics, and I plan to do more exercises like this in the future.
Here are some articles & documentaion I read along the way:
https://medium.com/@samia.khalid/bert-explained-a-complete-guide-with-theory-and-tutorial-3ac9ebc8fa7c
https://medium.com/nlplanet/two-minutes-nlp-beginner-intro-to-hugging-face-main-classes-and-functions-fb6a1d5579c4
https://theaisummer.com/hugging-face-vit/#:~:text=Because%20of%20the%20lack%20of%20a%20standardized%20training-loop,our%20own%20optimizers%2C%20losses%2C%20learning%20rate%20schedulers%2C%20etc.
https://medium.com/@sujathamudadla1213/difference-between-trainer-class-and-sfttrainer-supervised-fine-tuning-trainer-in-hugging-face-d295344d73f7
https://www.geeksforgeeks.org/what-is-huggingface-trainer/#
https://huggingface.co/docs/transformers/main_classes/trainer \