[TensorRT] IRuntime::deserializeCudaEngine: Error Code 6: API Usage Error (The engine plan file is generated on an incompatible device, expecting compute 8.9 got compute 8.6, please rebuild.) [TensorRT error] failed to deserialize engine

[bruceborgia]

[TensorRT] IRuntime::deserializeCudaEngine: Error Code 6: API Usage Error (The engine plan file is generated on an incompatible device, expecting compute 8.9 got compute 8.6, please rebuild.)
[TensorRT error] failed to deserialize engine
[ERR: InferenceWrapperTensorRT][44] Failed to load TensorRT model
NN Initialize fail

======================================
GPU：4090
CUDA：12.8
TENSORRT：10.8

加载Megatron_TRT时报错，这个怎么解决啊

[ceyxasm]

Your .engine / .plan file was built for a different GPU architecture.
4090 is actually SM 8.9 (Ada Lovelace)

So one of these is happening:

• You are NOT actually running on the 4090
• TensorRT is picking another GPU
• Engine was built with mismatched settings/container
• Multi-GPU system confusion

can u run
python3 -c "import torch; print(torch.cuda.get_device_name(0)); print(torch.cuda.get_device_capability(0))"
Ideally you should see (8,9)

Most likely fix:
Delete old TensorRT engines and rebuild them on the actual machine/GPU