Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Deepseek R1 multi-node example #754

Open
Tracked by #698
Jeffwan opened this issue Feb 26, 2025 · 0 comments
Open
Tracked by #698

Add Deepseek R1 multi-node example #754

Jeffwan opened this issue Feb 26, 2025 · 0 comments
Labels
area/distributed kind/documentation Improvements or additions to documentation priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete.

Comments

@Jeffwan
Copy link
Collaborator

Jeffwan commented Feb 26, 2025

🚀 Feature Description and Motivation

AIbrix provides the distributed inference feature and it work out of box with vLLM. We should add some guidances on full weights Deepseek-R1 deployment to attract people who like to use it.

Use Case

multi-node deployment

Proposed Solution

No response
@Jeffwan Jeffwan added kind/documentation Improvements or additions to documentation priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. area/distributed labels Feb 26, 2025
@Jeffwan Jeffwan mentioned this issue Feb 24, 2025
Open
41 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/distributed kind/documentation Improvements or additions to documentation priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Jeffwan