v2.0.0, update Autograd
- Now Autograd is tape-based, (previously it was fully recursive)
- Bug fixes and optimization
- Simplify
nn.Embedding
- Simplify autograd
reverse_broadcast
method - Fix
where
grad_fn method
- Simplify
- Add Seq2Seq Transformer example
- Other
- Add
nn.LogSoftmax
- Edit
nn.Linear bias initialization
- Оther fixes and improvements
- Add
New autograd version enables transformer training, overcoming previous limitations due to slow backpropagation
speed.