Skip to content
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
145 changes: 60 additions & 85 deletions docs/source/contributing/overview.md
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love the changes. Thanks!

Original file line number Diff line number Diff line change
@@ -1,150 +1,125 @@
# Contributing
# Contributing to vLLM-Spyre

Thank you for your interest in contributing to Spyre support on vLLM!

When submitting a PR, please make sure your code passes all linting checks:
There are several ways you can contribute:

## Linting
- Identify and report any issues or bugs.
- Suggest or implement new features.
- Improve documentation or contribute a how-to guide.

You can install the linting requirements using either `uv` or `pip`.
## Issues

Using `uv`:

```bash
uv sync --frozen --group lint --active --inexact
```

Using `pip`:

```bash
uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
```
If you encounter a bug or have a feature request, please search [existing issues](https://github.com/vllm-project/vllm-spyre/issues?q=is%3Aissue) first to see if it has already been reported. If not, please [create a new issue](https://github.com/vllm-project/vllm-spyre/issues/new/choose), providing as much relevant information as possible.

After installing the requirements, run the formatting script:

```bash
bash format.sh
```

Then, make sure to commit any changes made by the formatter:

```bash
git add .
git commit -s -m "Apply linting and formatting"
```

Now, you’re good to go! 🚀

## DCO and Signed-off-by

When contributing changes to this project, you must agree to the [DCO](https://github.com/vllm-project/vllm/blob/main/DCO).
Commits must include a `Signed-off-by:` header which certifies agreement with
the terms of the DCO.

Using `-s` with `git commit` will automatically add this header.
You can also reach out for support in the `#sig-spyre` channel in the [vLLM Slack](https://inviter.co/vllm-slack) workspace.

## Testing

### Running tests locally on CPU (No Spyre card)
### Testing Locally on CPU (No Spyre card)

1. (arm64 only) Install xgrammar
1. Install `xgrammar` (only for `arm64` systems):

:::{tip}
It's installed for x86_64 automatically.
`xgrammar` is automatically installed on `x86_64` systems.
:::

```sh
uv pip install xgrammar==0.1.19
```

1. (optional) Download `JackFram/llama-160m` model for tests
1. (Optional) Download the `JackFram/llama-160m` model:

```sh
python -c "from transformers import pipeline; pipeline('text-generation', model='JackFram/llama-160m')"
```

:::{caution}
Downloading the same model using HF API does not work locally on `arm64`.
The Hugging Face API download does **not** work on `arm64`.
:::

:::{tip}
:class: dropdown
We assume the model lands here:
By default, the model is saved to:

```sh
.cache/huggingface/hub/models--JackFram--llama-160m
```

:::

1. Source env variables needed for tests
1. Source environment variables:

```sh
source _local_envs_for_test.sh
```

1. (optional) Install dev dependencies (if vllm-spyre was installed without them)
1. (Optional) Install development dependencies:

```sh
uv pip install --group dev
```

1. Run the tests:
1. Run the tests using [markers](https://github.com/vllm-project/vllm-spyre/blob/main/pyproject.toml#L119):

```sh
python -m pytest -v -x tests -m "v1 and cpu and e2e"
```

### Continuous Batching Tests (CB)
### Testing Continuous Batching

:::{attention}
Temporary section until FMS custom branch is merged to main
Continuous batching currently requires the custom installation described below until the FMS custom branch is merged to main.
:::

Continuous batching requires a custom installation at the moment until the FMS custom branch is merged to main.
After completing the setup steps above:

To try it out, after following all steps for setting up testing as mentioned above,

1. Install custom FMS branch for CB:
1. Install custom FMS branch to enable support for continuous batching:

```sh
uv pip install git+https://github.com/foundation-model-stack/foundation-model-stack.git@paged_attn_mock --force-reinstall
```

#### Run only CB tests
2. Run the continuous batching tests:

```sh
python -m pytest -v -x tests/e2e -m cb
```

## Pull Requests

### Linting

When submitting a PR, please make sure your code passes all linting checks. You can install the linting requirements using either `uv` or `pip`.

Using `uv`:

```sh
python -m pytest -v -x tests/e2e -m cb
```bash
uv sync --frozen --group lint --active --inexact
```

## Debugging

We can debug using `debugpy` in VS code.

This is the content of the `launch.json` file which we need for debugging in VS code:

```sh
{
"version": "0.2.0",
"configurations": [
{
"name": "Python Debugger: local",
"type": "debugpy",
"request": "attach",
"connect": {
"host": "localhost",
"port": 5678
},
"justMyCode": false
}
]
}
Using `pip`:

```bash
uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
```

Run using
After installing the requirements, run the formatting script:

```sh
python -m debugpy --listen 5678 -m ...
```bash
bash format.sh
```

Then, make sure to commit any changes made by the formatter:

```bash
git add .
git commit -s -m "Apply linting and formatting"
```

### DCO and Signed-off-by

When contributing, you must agree to the [DCO](https://github.com/vllm-project/vllm-spyre/blob/main/DCO).Commits must include a `Signed-off-by:` header which certifies agreement with the terms of the DCO.

Using `-s` with `git commit` will automatically add this header.

## License

See <gh-file:LICENSE>.