Skip to content

A114-wrr-metric-names-for-computing-utilization.md

Latest commit

 

History

History
165 lines (119 loc) · 10.9 KB

A114-wrr-metric-names-for-computing-utilization.md

File metadata and controls

165 lines (119 loc) · 10.9 KB
Raw

A114: WRR Support for Custom Backend Metrics

  • Author(s): sauravzg
  • Approver: markdroth
  • Status: In review
  • Implemented in:
  • Last updated: 2026-01-30
  • Discussion at:

Abstract

This proposal updates the client-side weighted_round_robin (WRR) load balancing policy to support customizable utilization metrics. It adds a new configuration field metric_names_for_computing_utilization to the WRR LB policy config. This allows users to specify which backend metrics should be used to compute endpoint weights, enabling the use of custom metrics (via ORCA named metrics) instead of relying solely on the default application_utilization or cpu_utilization.

Background

The existing weighted_round_robin policy (defined in gRFC A58) calculates endpoint weights based on standard metrics provided by the backend via ORCA (Open Request Cost Aggregation) load reports. Specifically, it uses application_utilization if available, and falls back to cpu_utilization.

However, services may want to drive load balancing decisions based on other resources, such as memory utilization, queue depth, or custom application-defined metrics. The Custom Backend Metrics specification (ORCA) supports reporting arbitrary named metrics, and xDS has updated its WRR implementation to allow selecting these metrics for utilization calculation.

To support advanced load balancing scenarios, gRPC's WRR policy needs to support this flexibility.

Related Proposals

Proposal

Service Config Update

We will add a new field metric_names_for_computing_utilization to the WeightedRoundRobinLbConfig message in the Service Config.

message WeightedRoundRobinLbConfig {
  // ... existing fields ...

  // By default, Endpoint weight is computed by taking the max of the values of
  // the metric names specified here from the `OrcaLoadReport` proto.
  // For map fields in the ORCA proto, the string will be of the form
  // ``<map_field_name>.<map_key>``. For example, the string
  // ``named_metrics.foo`` will mean to look for the key ``foo`` in the
  // ORCA `named_metrics` field.
  // The first period separates the map field name from the key name, so
  // ``named_metrics.foo.bar`` references the key ``foo.bar``.
  // If none of the specified metrics are present in the load report, then
  // utilization will instead be computed based on the `application_utilization`
  // field reported by the endpoint.
  // If `application_utilization` is not set, then `cpu_utilization` is used
  // instead.
  repeated string metric_names_for_computing_utilization = 7;
}

Weight Calculation Logic

The weight calculation logic in the WRR policy will be updated to determine the utilization value as follows from the OrcaLoadReport

  1. Check Custom Metrics: If metric_names_for_computing_utilization is configured:
    • Iterate through the specified metric names.
    • Resolve Metric Value:
      • If the name is of the format field.key (e.g., named_metrics.foo), look up the map field field and retrieve the value for key.
      • If the name is a simple field name (e.g., cpu_utilization, mem_utilization), look up the field.
      • Only the following fields are supported:
        • application_utilization
        • cpu_utilization
        • mem_utilization
        • named_metrics.*
        • utilization.*
    • Compute Max: Track the maximum value among all successfully resolved, positive ( > 0), non-nan metrics.
    • If a max value is found, use it as the utilization.
  2. Fallback: If checking custom metrics did not determine a valid utilization value (or if metric_names_for_computing_utilization is not configured), fall back to the existing WRR utilization behavior defined in gRFC A58.

Pseudocode

function GetUtilization(report, configured_metrics):
  # 1. Check Custom Metrics
  max_util = null

  for metric_name in configured_metrics:
    value = null

    if metric_name contains ".":
      # Map lookup (e.g. "named_metrics.foo" -> map="named_metrics", key="foo")
      map_name, key = split_on_first_dot(metric_name)
      if report has map field map_name:
         value = report[map_name][key]
    else:
      # Root field lookup (e.g. "mem_utilization") via Reflection
      if report has field metric_name:
         value = report[metric_name]

    # Only consider valid, non-nan, positive values
    if value is not null and !is_nan(value) and value > 0:
      if max_util is null or value > max_util:
        max_util = value

  if max_util is not null:
    return max_util

  # 2. Fallback to existing WRR behavior (A58)
  if report.application_utilization > 0:
    return report.application_utilization
  return report.cpu_utilization

Implementation Notes

Since OrcaLoadReport is often exposed as a language-specific proxy object rather than a raw Protobuf message (e.g., in Java and C++), implementations are not expected to use Protobuf reflection to look up arbitrary fields. Instead, implementations should manually handle the supported metric list specified above.

As a consequence, support for any new standard fields added to OrcaLoadReport in the future will require explicit code changes in the implementation. This behavior is consistent with the current Envoy implementation.

Validity and Edge Cases

  • Nan Values: As shown above, NaN values in reports are explicitly ignored to prevent undefined behavior in weight calculations. This is relevant because behavior of max on NaN in c++ is inconsistent based on order.
  • Zero Values: Any value equal to 0.0 is treated as missing and ignored. This is consistent with gRFC A58 and the Envoy implementation, due to the lack of support for optional fields in the load report (where 0 cannot be distinguished from an unset field).
  • Negative Values: Any value < 0.0 is also treated as missing and ignored silently. This is consistent with the current Envoy implementation.
  • Bound Checks: The final selected utilization is subject to the standard validation logic from gRFC A58 (e.g., ensuring the value is positive) before being used to compute weight to avoid undefined behavior in weight calculations.
  • Normalization: The WRR policy does not normalize reported metrics; the application is responsible for this.

The rest of the weight calculation formula (using QPS, EPS, and penalty) from gRFC A58 remains unchanged.

xDS Integration

We will support the metric_names_for_computing_utilization field in the xDS ClientSideWeightedRoundRobin policy.

When converting the xDS configuration to the gRPC Service Config WeightedRoundRobinLbConfig, the metric_names_for_computing_utilization field should be copied over directly.

Temporary environment variable protection

The features described in this proposal will be guarded by the environment variable GRPC_EXPERIMENTAL_WRR_CUSTOM_METRICS, which defaults to false.

Rationale

  • Consistency with Envoy: This design mirrors the corresponding feature in Envoy, ensuring consistent behavior for xDS-controlled clients.
  • Flexibility: Allows users to define load balancing weights based on the actual bottleneck resource of their application (e.g., memory-bound services).
  • Backward Compatibility: The default behavior (using application_utilization or cpu_utilization) remains unchanged if the new field is not configured.

Implementation

This will be implemented in all languages C++, Java, and Go.

C++

Java

Go

  • xDS Integration: Update the configuration struct and conversion logic in converter.go to copy over the new field from the xDS configuration.
  • Config: Update the configuration struct in config.go to include metric_names_for_computing_utilization.
  • Weight Calculation Logic: Update the weight update function in balancer.go to implement the new utilisation selection logic.