Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore worker pods for gateway routing #776

Merged
merged 3 commits into from
Mar 3, 2025
Merged

Ignore worker pods for gateway routing #776

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
113533a
Ignore worker pods for gateway routing
varungup90 Mar 1, 2025
4500c19
ignore worker pods in UpdatePod as well
varungup90 Mar 3, 2025
2b6eb4c
use node worker as const
varungup90 Mar 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions pkg/cache/cache.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ type Block struct {

const (
modelIdentifier = "model.aibrix.ai/name"
nodeType = "ray.io/node-type"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am thinking if we want to add a aibrix label like orchestration.aibrix.ai/server-pod. autoscaling need that feature as well. We can start with ray specific label and replace with our own later as a separate issue

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, we have implement correctly in next iteration.

nodeWorker = "worker"
podPort = 8000
defaultPodMetricRefreshIntervalInMS = 50
expireWriteRequestTraceIntervalInMins = 10
Expand Down Expand Up @@ -296,6 +298,12 @@ func (c *Cache) addPod(obj interface{}) {
if !ok {
return
}
// ignore worker pods
nodeType, ok := pod.Labels[nodeType]
if ok && nodeType == nodeWorker {
klog.InfoS("ignored ray worker pod", "name", pod.Name)
return
}

c.Pods[pod.Name] = pod
c.addPodAndModelMappingLocked(pod.Name, modelName)
Expand Down Expand Up @@ -323,6 +331,20 @@ func (c *Cache) updatePod(oldObj interface{}, newObj interface{}) {
c.deletePodAndModelMapping(oldPod.Name, oldModelName)
}

// ignore worker pods
nodeType, ok := oldPod.Labels[nodeType]
if ok && nodeType == nodeWorker {
klog.InfoS("ignored ray worker pod", "name", oldPod.Name)
return
}

// ignore worker pods
nodeType, ok = newPod.Labels[nodeType]
if ok && nodeType == nodeWorker {
klog.InfoS("ignored ray worker pod", "name", newPod.Name)
return
}

// Add new mappings if present
if newOk {
c.Pods[newPod.Name] = newPod
Expand Down
Loading