Update the installation guidance and README.md #115
Conversation
| @@ -3,15 +3,26 @@ | |||
| Welcome to AIBrix, the foundational building blocks for constructing your own GenAI inference infrastructure. AIBrix offers a cloud-native solution tailored to meet the demands of enterprises aiming to deploy, manage, and scale LLMs efficiently. | |||
|
|
|||
| ## Key Features | |||
| TODO | |||
|
|
|||
| - High density Lora management | |||
There was a problem hiding this comment.
/cc @kr11 @brosoul @varungup90 please help review this part. what's the right description? any feedbacks?
There was a problem hiding this comment.
@Jeffwan Here are some drafted features about autoscaling, for your reference:
- Direct Metrics Access: Optimize scaler responsiveness with direct metrics retrieval from vLLM engine servers, bypassing traditional data pathways.
- Multiple Autoscaling Algorithms: Equip your infrastructure with HPA and KPA to ensure scalable and adaptable LLM deployment.
- Autoscaling Case Study: Provide actionable insights by examining autoscaling performance from experiments on practical LLM applications.
There was a problem hiding this comment.
For detailed features, we can list them in the feature intro section once this #2 is done
There was a problem hiding this comment.
here, we just a short description. any comments here?
| - High density Lora management | ||
| - Intelligent and LLM specific routing strategies | ||
| - LLM tailored pod autoscaler | ||
| - AI runtime sidecar (metrics merge, fast model downloading, admin operations) |
There was a problem hiding this comment.
fast
If we mention fast here, we may need to provide a comparison of its performance with some tools. But I don't think our current implementation (reusing SDK) has any reason to be faster.
The automatic selection of the number of download threads may be a breakthrough point. @Jeffwan
There was a problem hiding this comment.
Great. It's ok that we list some of the expected features here. We can adjust to more concrete words before release
* Update the README.md with features and installations * Comment the controllers needs dependencies
Pull Request Description
Related Issues
Resolves: #[Insert issue number(s)]
Important: Before submitting, please complete the description above and review the checklist below.
Contribution Guidelines (Expand for Details)
We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality, please adhere to the following guidelines:
Pull Request Title Format
Your PR title should start with one of these prefixes to indicate the nature of the change:
[Bug]: Corrections to existing functionality[CI]: Changes to build process or CI pipeline[Docs]: Updates or additions to documentation[API]: Modifications to aibrix's API or interface[CLI]: Changes or additions to the Command Line Interface[Misc]: For changes not covered above (use sparingly)Note: For changes spanning multiple categories, use multiple prefixes in order of importance.
Submission Checklist
By submitting this PR, you confirm that you've read these guidelines and your changes align with the project's contribution standards.