EleutherAI ML Scalability & Performance Reading Group Session 9, which builds on the TP strategy introduced in Megatron-LM with some additional techniques: sequence parallelism and selective activation recomputation.
Presenter: Daniel Vega-Myhre
| Name | Name | Last commit date | ||
|---|---|---|---|---|
parent directory.. | ||||
EleutherAI ML Scalability & Performance Reading Group Session 9, which builds on the TP strategy introduced in Megatron-LM with some additional techniques: sequence parallelism and selective activation recomputation.
Presenter: Daniel Vega-Myhre