Skip to content

Runtime error in windows, help me #2540

Open
@sjxhill

Description

@sjxhill

I have install TensorRT-LLM version: 0.15.0 in windows machine, GPU is RTX 2070 SUPER,
When run the sample:
from tensorrt_llm import LLM, SamplingParams
prompts = [
"Hello, my name is",
"The president of the United States is",
"The capital of France is",
"The future of AI is",
]
sampling_params = SamplingParams(temperature=0.8, top_p=0.95)

llm = LLM(model="TinyLlama-1.1B-Chat-v1.0/")
.....

a lot of runtime errors :

[TensorRT-LLM] TensorRT-LLM version: 0.15.0
Loading Model: [1/2] Loading HF model to memory
160it [00:02, 63.60it/s]
Time: 2.906s
Loading Model: [2/2] Building TRT-LLM engine
Unhandled exception caught in c10/util/AbortHandler.h
00007FFE12AE1DE400007FFE12AC2070 torch_python.dll!THPGenerator_initDefaultGenerator [ @ ]
00007FFFBA3BEE1200007FFFBA3BEDF0 ucrtbase.dll!terminate [ @ ]
00007FFF92361AAB00007FFF92361150 VCRUNTIME140_1.dll!_NLG_Return2 [ @ ]
00007FFF9236231700007FFF92361150 VCRUNTIME140_1.dll!_NLG_Return2 [ @ ]
00007FFF923640D900007FFF92364030 VCRUNTIME140_1.dll!_CxxFrameHandler4 [ @ ]
00007FFD84A3D55800007FFD8475FDF0 nvinfer_plugin_tensorrt_llm.dll!setLoggerFinder [ @ ]
00007FFFBCBD535F00007FFFBCBD5230 ntdll.dll!_chkstk [ @ ]
00007FFFBCB4E88600007FFFBCB4DDF0 ntdll.dll!RtlFindCharInUnicodeString [ @ ]
00007FFFBCB8495500007FFFBCB847C0 ntdll.dll!RtlRaiseException [ @ ]
00007FFFB9E7FE3C00007FFFB9E7FDD0 KERNELBASE.dll!RaiseException [ @ ]
00007FFF5CFE648000007FFF5CFE63F0 VCRUNTIME140.dll!CxxThrowException [ @ ]
00007FFDA1EF6BE700007FFDA1EF6B90 tensorrt_llm.dll!tensorrt_llm::common::throwRuntimeError [ @ ]
00007FFDA1F422A400007FFDA1F41EE0 tensorrt_llm.dll!tensorrt_llm::kernels::FusedMHARunnerV2::FusedMHARunnerV2 [ @ ]
00007FFD846E54D900007FFD846E52A0 nvinfer_plugin_tensorrt_llm.dll!tensorrt_llm::plugins::GPTAttentionPluginCommon::initialize [ @ ]
00007FFD846E681500007FFD846E66A0 nvinfer_plugin_tensorrt_llm.dll!tensorrt_llm::plugins::GPTAttentionPluginCommon::cloneImpl<tensorrt_llm::plugins::GPTAttentionPlugin> [ @ ]
00007FFF4E00B5B200007FFF4DFC8D10 nvinfer_10.dll!getInferLibMinorVersion [ @ ]
00007FFF4DF9441E00007FFF4DF545B0 nvinfer_10.dll!createInferBuilder_INTERNAL [ @ ]
00007FFEF606CDB9 tensorrt.cp310-win_amd64.pyd! [ @ ]
00007FFEF60170D6 tensorrt.cp310-win_amd64.pyd! [ @ ]
00007FFEF5F82EE8 tensorrt.cp310-win_amd64.pyd! [ @ ]
00007FFF5D019EEA00007FFF5D019E18 python310.dll!PyObject_IsTrue [ @ ]
00007FFF5D05FFBB00007FFF5D05FE58 python310.dll!PyObject_MakeTpCall [ @ ]
00007FFF5D19BF8F00007FFF5D17D114 python310.dll!Py_gitversion [ @ ]
00007FFF5D07729300007FFF5D0758A0 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FFF5D0749D700007FFF5D074950 python310.dll!PyFunction_Vectorcall [ @ ]
00007FFF5D05C00C00007FFF5D05BF54 python310.dll!PyVectorcall_Call [ @ ]
00007FFF5D05BE8700007FFF5D05BD44 python310.dll!PyObject_Call [ @ ]
00007FFF5D07B5E700007FFF5D0758A0 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FFF5D0749D700007FFF5D074950 python310.dll!PyFunction_Vectorcall [ @ ]
00007FFF5D05C00C00007FFF5D05BF54 python310.dll!PyVectorcall_Call [ @ ]
00007FFF5D05BE8700007FFF5D05BD44 python310.dll!PyObject_Call [ @ ]
00007FFF5D07B5E700007FFF5D0758A0 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FFF5D07361500007FFF5D0725C0 python310.dll!PyObject_GC_Malloc [ @ ]
00007FFF5D05C00C00007FFF5D05BF54 python310.dll!PyVectorcall_Call [ @ ]
00007FFF5D05BE8700007FFF5D05BD44 python310.dll!PyObject_Call [ @ ]
00007FFF5D07B5E700007FFF5D0758A0 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FFF5D0749D700007FFF5D074950 python310.dll!PyFunction_Vectorcall [ @ ]
00007FFF5D02A91700007FFF5D02A844 python310.dll!PyObject_FastCallDictTstate [ @ ]
00007FFF5D1381F400007FFF5D138178 python310.dll!PyObject_Call_Prepend [ @ ]
00007FFF5D13815000007FFF5D137064 python310.dll!PyBytesWriter_Resize [ @ ]
00007FFF5D05FFBB00007FFF5D05FE58 python310.dll!PyObject_MakeTpCall [ @ ]
00007FFF5D07C39F00007FFF5D0758A0 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FFF5D07361500007FFF5D0725C0 python310.dll!PyObject_GC_Malloc [ @ ]
00007FFF5D05C00C00007FFF5D05BF54 python310.dll!PyVectorcall_Call [ @ ]
00007FFF5D05BE8700007FFF5D05BD44 python310.dll!PyObject_Call [ @ ]
00007FFF5D07B5E700007FFF5D0758A0 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FFF5D0749D700007FFF5D074950 python310.dll!PyFunction_Vectorcall [ @ ]
00007FFF5D02A91700007FFF5D02A844 python310.dll!PyObject_FastCallDictTstate [ @ ]
00007FFF5D1381F400007FFF5D138178 python310.dll!PyObject_Call_Prepend [ @ ]
00007FFF5D13815000007FFF5D137064 python310.dll!PyBytesWriter_Resize [ @ ]
00007FFF5D05BF0300007FFF5D05BD44 python310.dll!PyObject_Call [ @ ]
00007FFF5D07B5E700007FFF5D0758A0 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FFF5D07862000007FFF5D0758A0 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FFF5D07361500007FFF5D0725C0 python310.dll!PyObject_GC_Malloc [ @ ]
00007FFF5D05C00C00007FFF5D05BF54 python310.dll!PyVectorcall_Call [ @ ]
00007FFF5D05BE8700007FFF5D05BD44 python310.dll!PyObject_Call [ @ ]
00007FFF5D07B5E700007FFF5D0758A0 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FFF5D07361500007FFF5D0725C0 python310.dll!PyObject_GC_Malloc [ @ ]
00007FFF5D05C00C00007FFF5D05BF54 python310.dll!PyVectorcall_Call [ @ ]
00007FFF5D05BE8700007FFF5D05BD44 python310.dll!PyObject_Call [ @ ]
00007FFF5D07B5E700007FFF5D0758A0 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FFF5D0749D700007FFF5D074950 python310.dll!PyFunction_Vectorcall [ @ ]
00007FFF5D02A91700007FFF5D02A844 python310.dll!PyObject_FastCallDictTstate [ @ ]
00007FFF5D1381F400007FFF5D138178 python310.dll!PyObject_Call_Prepend [ @ ]

Process finished with exit code -1073740791 (0xC0000409)

Metadata

Metadata

Labels

Generic RuntimeGeneral operational aspects of TRTLLM execution not in other categories.InvestigatingWindowstriagedIssue has been triaged by maintainers

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions