[BMG][OOB] hf_Albert and coat_lite_mini performance drop

[jianyizh]

🐛 Describe the bug

coat_lite_mini train with bs=128 performance regression from 100 to 152, hf_Albert train with bs=16 performance regression from 140 to 175. Due to pytorch/pytorch@82baae7

Versions

b580

[jianyizh]

this is expected because it add a cast. For example in output_code.py
before:

buf239 = workspace_10[0 * 256 * 768 : (0 + 1) * 256 * 768].view(256, 768).sum(dim=0)
triton_per_fused_add_sum_view_21.run(buf599, buf597, buf92, buf141, buf190, buf239, buf288, buf337, buf386, buf441, buf494, buf543, 768, 32, stream=stream0)

after:

buf239 = workspace_10[0 * 256 * 768 : (0 + 1) * 256 * 768].view(256, 768).sum(dim=0).to(torch.fp16)
triton_per_fused_add_sum_view_21.run(buf599, buf597, buf92, buf141, buf190, buf239, buf288, buf337, buf386, buf441, buf494, buf543, 768, 32, stream=stream0)