In task 1 we learnt how to fine-tune a language model and we saw how style transfer works. Conversations are a different beast however! In this task we will try our first approach at training a conversational model.
In this task we will try a naive approach to getting conversational style by simply feeding the model "raw" conversation data of the form:
<speaker1> Hi
<speaker2> Hey - how are you?
<speaker1> Great, thanks!
...
Our hope is that the model will simply learn this structure and we will be able to query the model with an input of the form:
<speaker2> Am I speaking to a bot?
<speaker1>
We then expect the model to extend the text from this prefix.
You are free to either use either your own chat logs or conversational data from other sources.
In order to obtain your chat logs you need to first request them and then parse them into a format we can use. For this we will use the Chatistics repository.
Do the following steps on your local machine:
- Download the raw data and place them in the appropriate folder
- Parse the chat logs (this will generate pickled Dataframes)
- Export and filter the data you wish to use for this workshop. For an overview of the filter options run
python export.py --help. Once decided run the following command to export the data to JSON
python export.py -f json <your filter options>
- Place this JSON file now into the folder
2/data/.
You can either run this code locally or use the Colab notebook.
- Clone this repository using
git clone [email protected]:mar-muel/artificial-self-AMLD-2020.git && cd artificial-self-AMLD-2020
- Install the dependencies Make sure you have PyTorch with GPU-support installed. Follow the instructions here here to install the proper version depending on your OS. Then run:
cd 2
pip install -r requirements.txt
- Make sure you have a single JSON file in the folder
2/data/(otherwise specify the--data_pathargument) and run
python train.py --run_name run1
- Interact with the model by running
python interact.py --run_name run1
