Proposal: Support Descheduler Extender Plugin

[googs1025]

• To enhance flexibility, Descheduler can support an Extender plugin (like upstream scheduler) allowing external services to make decision on whether a pod can be evicted. This enables integration with custom logic without modifying Descheduler code.

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: default
    pluginConfig:
      - ...
      - name: "ExternalDecision"
        args:
          # for the external HTTP service (Extender)
          extender:
            # Base URL of the external service
            urlPrefix: "https://127.0.0.1:8080"
            # endpoint path to call (e.g., POST <urlPrefix>/<decisionVerb>)
            decisionVerb: "evict" 
            # Whether to ignore failures (timeout, network error, non-200 response)
            ignorable: true                   # Required: true/false
            # Timeout for the HTTP request
            httpTimeout: 5s                   # Required: e.g., "5s"
            # Enable HTTPS
            enableHTTPS: true                 # Optional: defaults to false
            # TLS configuration for verifying the server
            tlsConfig:
              caCertFile: "/path/to/ca.crt"   # Optional: path to CA certificate

            # Plugin-specific behavior
            # when the extender call fails (ignored if ignorable=true)
            failPolicy: "Deny"                  # Optional: "Allow", "Deny", "Ignore" (default: "Deny")

Request:
Endpoint: POST /
Request Body (JSON):

{
  "pod": { /* Full v1.Pod object */ },
  "node": { /* Full v1.Node object */ }
}

Response Body (JSON):

{
  "allow": true,     // boolean: true to allow eviction, false to block
  "reason": "Pod is eligible for cost-saving migration."
}

Plugin Behavior

• Success (allow: true): Eviction pod.
• Blocked (allow: false): Pod is skipped; reason is logged.
• Failure (timeout, error, non-200):
  • If ignorable: true: Proceed as if allow: true (or respect failPolicy if implemented).
  • If ignorable: false: Block eviction (allow: false), unless failPolicy overrides.

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Describe alternatives you've considered

What version of descheduler are you using?

descheduler version:

Additional context

[googs1025]

some golang struct can be defined:

• Descheduler Extender Interface

// Extender holds configuration for one external service.
type Extender struct {
    URLPrefix     string
    DecisionVerb  string
    Ignorable     bool
    HTTPTimeout   time.Duration
    EnableHTTPS   bool
    TLSConfig     *TLSConfig
    HTTPHeaders   map[string]string
}

// EvictionRequest is sent to the external service.
type EvictionRequest struct {
    Pod     v1.Pod            `json:"pod"`
    Node    v1.Node           `json:"node"`
}

// EvictionResponse is received from the external service.
type EvictionResponse struct {
    Allow  bool   `json:"allow"`
    Reason string `json:"reason"`
}

• Extender Client

// Plugin is the parent type for all descheduling framework plugins.
type Plugin interface {
    Name() string
}

// ExtenderDecisionPlugin is the interface for the ExternalDecision plugin.
type ExtenderDecisionPlugin interface {
    Plugin
    // DecideEviction asks external services if a pod can be evicted.
    // Returns:
    // - allowed: true if eviction is permitted.
    // - status: nil if success, or a Status with error if call failed.
    DecideEviction(ctx context.Context, pod *v1.Pod, node *v1.Node) (allowed bool, status *Status)
}

[googs1025]

func (d *ExternalDecision) Deschedule(ctx context.Context, nodes []*v1.Node) *frameworktypes.Status {
         ...

	// Build node map and collect candidate pods
	for _, node := range nodes {
		logger.V(2).Info("Processing node", "node", klog.KObj(node))
		pods, err := podutil.ListAllPodsOnANode(node.Name, d.handle.GetPodsAssignedToNodeFunc(), d.podFilter)
		if err != nil {
			return &frameworktypes.Status{
				Err: fmt.Errorf("error listing pods on node %s: %v", node.Name, err),
			}
		}

		nodeMap[node.Name] = node
		podsToEvict = append(podsToEvict, pods...)
	}


	for _, pod := range podsToEvict {
		node, exists := nodeMap[pod.Spec.NodeName]
		if !exists {
			logger.V(2).Info("Node not found in map, skipping", "pod", klog.KObj(pod), "node", pod.Spec.NodeName)
			continue
		}

		// call extender
		allow, reason, err := d.extenderClient.DecideEviction(ctx, pod, node)
		if err != nil {
			// Handle extender failure based on `ignorable` and `failPolicy`
			logger.Error(err, "Failed to get eviction decision from external service", "pod", klog.KObj(pod))
			switch d.failPolicy {
			case "Allow", "":
				...
			case "Ignore":
				continue loop
			case "Deny":
				continue loop
			}
		}

		if !allow {
			...
		}


		evictErr := d.handle.Evictor().Evict(ctx, pod, evictions.EvictOptions{StrategyName: ExternalDecisionPluginName})
		if evictErr == nil {
			continue
		}

		// Handle eviction limits
		switch evictErr.(type) {
		case *evictions.EvictionNodeLimitError:
			continue loop
		case *evictions.EvictionTotalLimitError:
			return nil
		default:
			logger.Error(evictErr, "Eviction failed after external approval", "pod", klog.KObj(pod))
		}
	}

	return nil
}

[googs1025]

cc @ingvagabund @ricardomaraschini

[googs1025]

https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/extender.go

We can refer to the scheduler extender code for a large part of this. I'm not sure if a similar solution 🤔 has been discussed before. If it's feasible, I'd be happy to give it a try.

[googs1025]

If we consider the time latency (http base), I think this is a problem, but it seems that it needs to be decided later. After all, the scheduler provides scheduling in seconds, but descheduler seems to be a similar time task. The two are not quite the same.

[googs1025]

just some Implementation Path in #1727

[googs1025]

If we consider transmission performance, maybe we can only transmit pod namespace name and let the server implement something like informer to obtain the pod node by itself?

Descheduler (client)

↓ POST 

{"podName": "pod-a", "podNamespace": "default"}

External Decision Server (server)

→ Fetch Pod and Node objects using SharedInformer

→ Execute business decision logic

→ Return {"allow": true, "reason": "..."}

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[ingvagabund]

Hello @googs1025.

I finally found some time to review your ideas. Thank you for sharing a lot of details here.

allowing external services to make decision on whether a pod can be evicted.

If the goal here is the eviction rules it's sufficient to create a new plugin that implements the EvictionPlugin interface:

type EvictorPlugin interface {
	Plugin
	Filter(pod *v1.Pod) bool
	PreEvictionFilter(pod *v1.Pod) bool
}

This way each profile can decide whether the external consulting is enabled.

[ingvagabund]

If we consider the time latency (http base), I think this is a problem, but it seems that it needs to be decided later. After all, the scheduler provides scheduling in seconds, but descheduler seems to be a similar time task. The two are not quite the same.

In the descheduler case this is not such an issue since if the Filter extension point gets implemented the relevant pods are usually evicted at the end of each plugin after all pods are processed.