Experiment with pre-trained model to build Text Summarization Model for summarize Indonesia Article (in this project will be using dataset id_liputan6 by Fajri Koto)
- Used Pre-Trained Model : indolem/indobert-base-uncased source
- Used Dataset : id_liputan6 by Fajri Koto
- Finetuning Parameter :
- batch_size = 8
- learning_rate = 2e-5
- epoch = 8 (with early stoping)
- weight_decay = 0.01
- Finetuning Evaluation :
- Rouge1 : 0.33 (+/- 0.14)
- Rouge2 : 0.15 (+/- 0.12)
- RougeL : 0.28 (+/- 0.13)
- RougeLsum : 0.28 (+/- 0.13)