-
Notifications
You must be signed in to change notification settings - Fork 26
👷 use UV package manager #63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Joe Runde <[email protected]>
|
👋 Hi! Thank you for contributing to vLLM support on Spyre. Now you are good to go 🚀 |
Signed-off-by: Joe Runde <[email protected]>
Signed-off-by: Joe Runde <[email protected]>
| ] | ||
|
|
||
| [tool.uv.sources] | ||
| vllm = { git = "https://github.com/vllm-project/vllm", rev = "v0.8.0" } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Specifying the source for vllm allows uv to run the install from source, specifying the empty target so that the gpu deps don't get installed
Signed-off-by: Joe Runde <[email protected]>
Signed-off-by: Joe Runde <[email protected]>
Signed-off-by: Joe Runde <[email protected]>
Signed-off-by: Joe Runde <[email protected]>
Signed-off-by: Joe Runde <[email protected]>
Signed-off-by: Joe Runde <[email protected]>
Signed-off-by: Joe Runde <[email protected]>
| dependencies = [ | ||
| "fms-model-optimizer>=0.2.0", | ||
| "ibm-fms==0.0.8", | ||
| "vllm", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can specify vllm and fms-model-optimizer, and don't have to worry about either of them overriding torch 👍
| ln -sf $(which pip${PYTHON_VERSION}) /usr/bin/pip | ||
|
|
||
| # Download and install vllm ########################################################### | ||
| RUN git clone --depth 1 https://github.com/vllm-project/vllm.git \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no more installing vllm manually 😎😎😎
|
@tdoublep I put all the code linting checks together into one job, so if we want to merge this we can remove the requirement on the |
| if: always() | ||
| run: codespell --toml pyproject.toml |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we also add mypy here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could, though right now that's running on multiple Python versions. I assume that's to make sure that mypy will be happy with any version of Python that somebody wants to develop locally with. But I guess it's also not too hard to just run all these checks with every Python version as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we really need to run it with every python version, this is just inherited from upstream checks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Fixes #61
This integrates our project with
uv, so that we can cleanly reject torch upgrades while installing all dependencies. This also modernizes ourpyproject.tomlto contain the source of truth for our dependencies, and sorts them into groups.The updates to the Dockerfile demonstrate this working, though I think the dockerfile may not be around for much longer.
This blast radius ended up a bit bigger than I originally intended, but...
yapfrelied onlib2to3which no longer exists in newer pythons, and has to be installed separately on the system. This broke the simpleruvsetup flow on github actions, so It was easier to just upgrade. This caused some changes in formatting in the test filessetup.pyfile pretty much only existed to parse those, so it was removed as well. This required a couple mode edits topyproject.tomlto specify setuptools as the build backend and set the extra entrypoint upWe can now either use
uvdirectly to install the plugin and manage our local dev environments, or haveuvcompile the dependencies back into requirements files to emulate the old pip-only workflow. I've updated the pr comment action to provide instructions for both options of installing the linting dependencies.