[BMG][OOB] t5 inference performance drop due to triton commit pin update #2653
Description
🐛 Describe the bug
guity commit is triton update pytorch/pytorch@5d1459a
| model | bs | perf before | perf after |
|---|---|---|---|
| hf_T5 | 16 | 353 | 409 |
| hf_T5_base | 1 | 69 | 73 |
| hf_T5_generate | 16 | 121 | 129 |
| T5ForConditionalGeneration | 32 | 246 | 292 |
| T5Small | 32 | 246 | 292 |
Versions
b580