Support Pytorch AOTInductor

[zeruniverse]

Is your feature request related to a problem? Please describe.

Current Pytorch backend uses libtorch, for which Pytorch community no longer actively develop / maintain. Experimental 2.0 backend uses python, which:

• creates a separate process for each model instances and cannot share cuda contexts, using a lot of GPU memory and making the management of GPU memory difficult.
• is slow due to python

Describe the solution you'd like
A clear and concise description of what you want to happen.

Make pytorch backend support AOTInductor. This supports cpp inferences and is the recommended way from pytorch community.

https://docs.pytorch.org/docs/2.8/torch.compiler_aot_inductor.html

[dzier]

PyTorch 2.0 support in the PyTorch backend has not been experimental for a long time and fully supports Torch.Compile. The documentation will be updated in the the 25.10 release of Triton to reflect this.

We have started to look into AOT Inductor support, however, AOT Inductor is, itself, experimental. We will monitor and update this issue as we plan AOT Inductor support.

[zeruniverse]

PyTorch 2.0 support in the PyTorch backend has not been experimental for a long time and fully supports Torch.Compile. The documentation will be updated in the the 25.10 release of Triton to reflect this.We have started to look into AOT Inductor support, however, AOT Inductor is, itself, experimental. We will monitor and update this issue as we plan AOT Inductor support.

Thanks for reply!

Triton currently uses Pytorch 2.9.0a0+50eac811a6 for which TorchScript is already marked deprecated and the AOTInductor is marked as prototype status. However, Pytorch 2.10.0 removed this warning which makes AOTI a stable feature. Given the history of Pytorch release cycle and how Triton tracks with Pytorch, Triton 25.12 will ship with Pytorch 2.10.0. So I think it's a good time to start AOTI support development and add it to 25.12 release.

Regarding current Pytorch 2.0 support, it uses Python backend. I currently serve more than 30 models using TorchScript. It's not possible to do it with Python since each model instance is a separate process and that causes CUDA OOM.