Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Initial implementation of radix tree-based cache (#678)
* Initial implementation of radix tree-based cache * Adding copyright * Updated tree.go. GPU -> Pod * Migrated tree.go from gangmuk/prefix_and_load_aware_routing branch * Changed to vllm/aibrix and Removed updatedMapping argument in AddPrefix * Removed unused functions (Detokenizeation) * Updated import utils to vllm/aibrix * Updated AddPrefix to handle modeltopod and updating all related nodes together * Updated evictNode and MatchPrefix to pass test * Updated matchPrefixHelper to return empty list not nil * Moved up LPRadixCache definition before function definitions --------- Co-authored-by: Gangmuk <[email protected]>
- Loading branch information