Skip to content

feat(model):support qwen3 dense in trt flow - #5350

Closed
xq25478 wants to merge 1 commit into
NVIDIA:mainfrom
xq25478:support_trt_flow_qwen3_dense
Closed

xq25478 wants to merge 1 commit into
NVIDIA:mainfrom
xq25478:support_trt_flow_qwen3_dense

Conversation

@xq25478

@xq25478 xq25478 commented Jun 19, 2025 •

Copy link
Copy Markdown
Contributor

feat(model) support qwen3 dense in trt flow

@xq25478
xq25478 force-pushed the support_trt_flow_qwen3_dense branch from ca26585 to eaa3a7c Compare June 19, 2025 04:44
@xq25478

xq25478 commented Jun 19, 2025

Copy link
Copy Markdown
Contributor Author

/bot run

@poweiw poweiw added the Community want to contribute PRs initiated from Community label Jun 24, 2025
@Fan-Yunfan

Fan-Yunfan commented Jul 23, 2025 •

Copy link
Copy Markdown
Contributor

Hello,When I use above workflow, I encountered the following problem:

v0.15.0/tensorrt_llm/models/qwen/convert.py", line 535, in convert_hf_qwen
assert (k_bias.shape[0] % (mapping.tp_size * head_size)) == 0
AttributeError: 'NoneType' object has no attribute 'shape'

My model is Qwen3-0.6B,and trtllm version is v0.15.0. Have you encountered the above error?
I tried migrating the conversion code for Qwen3 TRT from the main branch to TRT-LLM v0.15.0, but the aforementioned error still occurs.

@xq25478

xq25478 commented Jul 24, 2025

Copy link
Copy Markdown
Contributor Author

Hello,When I use above workflow, I encountered the following problem:

v0.15.0/tensorrt_llm/models/qwen/convert.py", line 535, in convert_hf_qwen assert (k_bias.shape[0] % (mapping.tp_size * head_size)) == 0 AttributeError: 'NoneType' object has no attribute 'shape'

My model is Qwen3-0.6B,and trtllm version is v0.15.0. Have you encountered the above error? I tried migrating the conversion code for Qwen3 TRT from the main branch to TRT-LLM v0.15.0, but the aforementioned error still occurs.

please use latest trtllm version,thank you。

@xq25478 xq25478 closed this Jul 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Community want to contribute PRs initiated from Community

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants