Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Platform] add dispatch key #17

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[Platform] add dispatch key #17

wants to merge 1 commit into from

Conversation

MengqingCao
Copy link
Contributor

What this PR does / why we need it?

Add dispatch key for NPU, so that the log could be print correctly.

Now

executor_base.py:110] # CPU blocks: 220478, # CPU blocks: 21845

After this pr

executor_base.py:110] # NPU blocks: 220478, # CPU blocks: 21845

Does this PR introduce any user-facing change?

N/A

How was this patch tested?

CI passed and log printed as above

@MengqingCao MengqingCao marked this pull request as draft February 7, 2025 06:33
@wangxiyuan
Copy link
Collaborator

wangxiyuan commented Feb 17, 2025 •
edited
Loading

from torch expert suggestion: change the value to PrivateUse1.

@MengqingCao
[Platform] add dispatch key
747a275 
Signed-off-by: MengqingCao <[email protected]>
@MengqingCao
Copy link
Contributor Author

from torch expert suggestion: change the value to PrivateUse1.

Thanks a lot! This could fix the issue that torch does not recognize key npu.
But the log printed will become executor_base.py:110] # PrivateUse1 blocks: 220478, # CPU blocks: 21845, I'll fix this in vLLM.

@MengqingCao MengqingCao marked this pull request as ready for review February 17, 2025 02:53
@wangxiyuan
Copy link
Collaborator

from torch expert suggestion: change the value to PrivateUse1.

Thanks a lot! This could fix the issue that torch does not recognize key npu. But the log printed will become executor_base.py:110] # PrivateUse1 blocks: 220478, # CPU blocks: 21845, I'll fix this in vLLM.

Yes, the log in vllm should use device_name instead.

@MengqingCao
Copy link
Contributor Author

Yes, the log in vllm should use device_name instead.

Agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wangxiyuan wangxiyuan wangxiyuan approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@MengqingCao @wangxiyuan