Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 66 additions & 83 deletions docs/source/contributing/overview.md
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love the changes. Thanks!

Original file line number Diff line number Diff line change
Expand Up @@ -2,89 +2,55 @@

Thank you for your interest in contributing to Spyre support on vLLM!

When submitting a PR, please make sure your code passes all linting checks:
There are several ways you can contribute:

## Linting
- Identify and report any issues or bugs.
- Suggest or implement new features.
- Improve documentation or contribute a how-to guide.

You can install the linting requirements using either `uv` or `pip`.
## Issues

Using `uv`:

```bash
uv sync --frozen --group lint --active --inexact
```
If you encounter a bug or have a feature request, please search [existing issues](https://github.com/vllm-project/vllm-spyre/issues?q=is%3Aissue) first to see if it has already been reported. If not, please [create a new issue](https://github.com/vllm-project/vllm-spyre/issues/new/choose), providing as much relevant information as possible.

Using `pip`:

```bash
uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
```

After installing the requirements, run the formatting script:

```bash
bash format.sh
```

Then, make sure to commit any changes made by the formatter:

```bash
git add .
git commit -s -m "Apply linting and formatting"
```

Now, you’re good to go! 🚀

## DCO and Signed-off-by

When contributing changes to this project, you must agree to the [DCO](https://github.com/vllm-project/vllm/blob/main/DCO).
Commits must include a `Signed-off-by:` header which certifies agreement with
the terms of the DCO.

Using `-s` with `git commit` will automatically add this header.
You can also reach out for support in the `#sig-spyre` channel in the [vLLM Slack](https://inviter.co/vllm-slack) workspace.

## Testing

### Running tests locally on CPU (No Spyre card)
### Testing Locally on CPU (No Spyre card)

1. (arm64 only) Install xgrammar
1. Install `xgrammar` (only for `arm64` systems):

:::{tip}
It's installed for x86_64 automatically.
`xgrammar` is automatically installed on `x86_64` systems.
:::

```sh
uv pip install xgrammar==0.1.19
```

1. (optional) Download `JackFram/llama-160m` model for tests
1. (Optional) Download the `JackFram/llama-160m` model:

```sh
python -c "from transformers import pipeline; pipeline('text-generation', model='JackFram/llama-160m')"
```

:::{caution}
Downloading the same model using HF API does not work locally on `arm64`.
The Hugging Face API download does **not** work on `arm64`.
:::

:::{tip}
:class: dropdown
We assume the model lands here:
By default, the model is saved to:

```sh
.cache/huggingface/hub/models--JackFram--llama-160m
```

:::

1. Source env variables needed for tests
1. Source environment variables:

```sh
source _local_envs_for_test.sh
```

1. (optional) Install dev dependencies (if vllm-spyre was installed without them)
1. (Optional) Install development dependencies:

```sh
uv pip install --group dev
Expand All @@ -96,55 +62,72 @@ Using `-s` with `git commit` will automatically add this header.
python -m pytest -v -x tests -m "v1 and cpu and e2e"
```

### Continuous Batching Tests (CB)
Here are a list of `pytest` markers you can use to filter tests:

:::{literalinclude} ../../../pyproject.toml
:start-after: begin-test-markers-definition
:end-before: end-test-markers-definition
:language: python
:::

### Testing Continuous Batching

:::{attention}
Temporary section until FMS custom branch is merged to main
Continuous batching currently requires the custom installation described below until the FMS custom branch is merged to main.
:::

Continuous batching requires a custom installation at the moment until the FMS custom branch is merged to main.

To try it out, after following all steps for setting up testing as mentioned above,
After completing the setup steps above:

1. Install custom FMS branch for CB:
1. Install custom FMS branch to enable support for continuous batching:

```sh
uv pip install git+https://github.com/foundation-model-stack/foundation-model-stack.git@paged_attn_mock --force-reinstall
```

#### Run only CB tests
2. Run the continuous batching tests:

```sh
python -m pytest -v -x tests/e2e -m cb
```

## Pull Requests

### Linting

When submitting a PR, please make sure your code passes all linting checks. You can install the linting requirements using either `uv` or `pip`.

Using `uv`:

```bash
uv sync --frozen --group lint --active --inexact
```

Using `pip`:

```sh
python -m pytest -v -x tests/e2e -m cb
```bash
uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
```

## Debugging

We can debug using `debugpy` in VS code.

This is the content of the `launch.json` file which we need for debugging in VS code:

```sh
{
"version": "0.2.0",
"configurations": [
{
"name": "Python Debugger: local",
"type": "debugpy",
"request": "attach",
"connect": {
"host": "localhost",
"port": 5678
},
"justMyCode": false
}
]
}
After installing the requirements, run the formatting script:

```bash
bash format.sh
```

Run using
Then, make sure to commit any changes made by the formatter:

```sh
python -m debugpy --listen 5678 -m ...
```bash
git add .
git commit -s -m "Apply linting and formatting"
```

### DCO and Signed-off-by

When contributing, you must agree to the [DCO](https://github.com/vllm-project/vllm-spyre/blob/main/DCO).Commits must include a `Signed-off-by:` header which certifies agreement with the terms of the DCO.

Using `-s` with `git commit` will automatically add this header.

## License

See <gh-file:LICENSE>.
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,8 @@ skip_gitignore = true

[tool.pytest.ini_options]
pythonpath = ["."]

# begin-test-markers-definition
markers = [
"skip_global_cleanup",
"e2e: Tests using end-to-end engine spin-up",
Expand All @@ -131,6 +133,7 @@ markers = [
"utils: Tests for utility functions",
"worker: Tests for worker logic",
]
# end-test-markers-definition

[tool.pymarkdown]
plugins.md013.enabled = false # line-length
Expand Down