Skip to content

Commit

Permalink
Initial implementation of radix tree-based cache (#678)
Browse files Browse the repository at this point in the history
* Initial implementation of radix tree-based cache

* Adding copyright

* Updated tree.go. GPU -> Pod

* Migrated tree.go from gangmuk/prefix_and_load_aware_routing branch

* Changed to vllm/aibrix and Removed updatedMapping argument in AddPrefix

* Removed unused functions (Detokenizeation)

* Updated import utils to vllm/aibrix

* Updated AddPrefix to handle modeltopod and updating all related nodes together

* Updated evictNode and MatchPrefix to pass test

* Updated matchPrefixHelper to return empty list not nil

* Moved up LPRadixCache definition before function definitions

---------

Co-authored-by: Gangmuk <[email protected]>
  • Loading branch information
gangmuk and Gangmuk authored Feb 20, 2025
1 parent 2dd650a commit 728c7c4
Show file tree
Hide file tree
Showing 2 changed files with 603 additions and 0 deletions.
Loading

0 comments on commit 728c7c4

Please sign in to comment.