[2023/05/06] Significant additions and changes to the repository's code base

[Cryolite]

Significant additions and changes have been made to the repository's code base. The main content is as follows:

• Backward incompatibility of annotation data for offline reinforcement learning: Due to the reorganization of the training data format for offline reinforcement learning, annotation data created for the old code base can no longer be used for the new code base. From now on, in order to create annotation data for offline reinforcement learning, use annotate4rl.py.
• Backward incompatibility of models: Due to the reorganization of the model structures, models trained with the old codebase can no longer be retrained or fine-tuned with the new codebase. If you want to migrate a model trained with the old codebase to the new codebase, please let me know. I will handle it individually.
• Implemented a new model which is a variant of IQL where the Q-function model and V-function model share parameters (as used in ILQL).
• Options for training programs are now specified within the Hydra framework.
• Added support for position embedding to encode positional information of embeddings for BERT.
• Significantly increased simulation speed (~x4) by batching inference.
• Implemented a new feature called reward plug-in. With this, users can freely design rewards in reinforcement learning, allowing them to adjust the direction of the model's training in reinforcement learning.