Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update documentation for Quick Start and Base Model deployment #745

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update documentation for Quick Start and Base Model deployment #745

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,13 @@ The initial release includes the following key features:

## Quick Start

To get started with AIBrix, clone this repository and follow the setup instructions in the documentation. Our comprehensive guide will help you configure and deploy your first LLM infrastructure seamlessly.
To get started with AIBrix, clone this repository and follow the setup instructions in the [documentation](https://aibrix.readthedocs.io/latest/getting_started/quickstart.html), which will help you configure and deploy your first LLM infrastructure seamlessly.

```shell
# Prerequisites:
# After installing Docker and Kubernetes, please make sure your Kubernetes cluster ready, and then run the following commands to install AIBrix components in your Kubernetes cluster.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically, docker is not necessary, people may need other container runtimes like containerd or cri-o. After getting Kubernetes ready should be good. container runtime is prerequsite to run kubernetes. Most of the users will use public cloud provides kubernetes in prod, there's no need to install it.



# Local Testing
git clone https://github.com/vllm-project/aibrix.git
cd aibrix
Expand Down
2 changes: 1 addition & 1 deletion docs/source/getting_started/installation/installation.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ Nightly Version
kubectl create -k config/default


Install AIBrix in testing Environments
Install AIBrix in Testing Environments
--------------------------------------

.. toctree::
Expand Down
9 changes: 6 additions & 3 deletions docs/source/getting_started/quickstart.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Quickstart
Install AIBrix
^^^^^^^^^^^^^^

Get your kubernetes cluster ready, run following commands to install aibrix components in your cluster.
Install Docker and Kubernetes, get your Kubernetes cluster ready, and run the following commands to install AIBrix components in your Kubernetes cluster.

.. note::
If you just want to install specific components or specific version, please check installation guidance for more installation options.
Expand All @@ -30,9 +30,12 @@ Wait for few minutes and run `kubectl get pods -n aibrix-system` to check pod st
aibrix-redis-master-84769768cb-fsq45 1/1 Running 0 5h24m


Deploy base model
Deploy Base Model
^^^^^^^^^^^^^^^^^

.. note::
In the given example, please ensure that your node is equipped with NVIDIA GPUs. Otherwise, the pod will stuck in pending state.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most users will expect to run this on GPU env. I think here we do not want to implicitly mention NVIDIA GPUs is needed. Technically, amd gpus or other NPUs works as well. we can say device plugin for accelerator is needed.


Save yaml as `model.yaml` and run `kubectl apply -f model.yaml`.

.. literalinclude:: ../../../samples/quickstart/model.yaml
Expand All @@ -44,7 +47,7 @@ Ensure that:
2. The `--served-model-name` argument value in the `Deployment` command is also consistent with the `Service` name and `model.aibrix.ai/name` label.


Invoke the model endpoint using gateway api
Invoke the Model Endpoint via Gateway API
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Depending on where you deployed the AIBrix, you can use either of the following options to query the gateway.
Expand Down