Compatible with AWS H100 p5 instances

[rizhashmi]

I acknowledge that all tests were conducted on an A100 and other hardware, but may I know if this code/model is compatible with an H100?

[rwightman]

There shouldn't be any noteworth differencecs between running as is on H100 vs A100 aside from increased speed on the H100.

However, not utilizing any FP8 modes on the H100 for further gains, but that would require additional code, careful testing, etc for stable training and good results.