Trying to run torch_compile_with_torchao cannot import name 'TorchAoConfig' from 'transformers'

[lukaLLM]

Hi, while running the notebook from recipes https://github.com/huggingface/huggingface-llama-recipes/blob/main/torch_compile_with_torchao.ipynb I got cannot import name 'TorchAoConfig' from 'transformers' I installed newest pytorch using conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch-nightly -c nvidia and followed the installation here https://github.com/pytorch/ao am I doing smth wrong ? I use Meta-Llama-3.1-8B-Instruct locally.

[ariG23498]

Hey @lukaLLM

Could you give https://github.com/huggingface/huggingface-llama-recipes/blob/main/performance_optimization/torch_compile_with_torchao.ipynb another try, with the latest installation of transformers?

[lukaLLM]

I will need some this as I get dependency complication in project if I do that. I think I should also configure it diffrently on windows. Still not sure if its faster than jsut flash attention 2.