Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

api: adds cost specifier to RateLimitRule #4957

Merged
merged 14 commits into from
Jan 9, 2025
86 changes: 86 additions & 0 deletions api/v1alpha1/ratelimit_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ type LocalRateLimit struct {
//
// +optional
// +kubebuilder:validation:MaxItems=16
// +kubebuilder:validation:XValidation:rule="self.all(foo, !has(foo.cost.response))", message="response cost is not supported for Local Rate Limits"
zirain marked this conversation as resolved.
Show resolved Hide resolved
Rules []RateLimitRule `json:"rules"`
}

Expand Down Expand Up @@ -91,6 +92,91 @@ type RateLimitRule struct {
// 429 HTTP status code is sent back to the client when
// the selected requests have reached the limit.
Limit RateLimitValue `json:"limit"`
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like it is still required to specify the number of requests limit, how we can handle the case when we only care of the cost not the number of the request limit?

Copy link
Member Author

@mathetake mathetake Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok so first of all, this limit is nothing to do with that purpose. And to do the limit purely on the token number (in this context response cost only), you can simply set the cost.response = {from: Number, Number: 0} as per the comment - zero can be used to "only check the budget and if not left anything, then reject".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also with this new cost field, limit.requests doesnt quite fit anymore, something like limit.total feels better and can be an alias to limit.requests in the future to improve defining intent

// Cost specifies the cost of requests and responses for the rule.
//
// This is optional and if not specified, the default behavior is to reduce the rate limit counters by 1 on
// the request path and do not reduce the rate limit counters on the response path.
//
// +optional
// +notImplementedHide
Cost *RateLimitCost `json:"cost,omitempty"`
}

type RateLimitCost struct {
// Request specifies the number to reduce the rate limit counters
// on the request path. If this is not specified, the default behavior
// is to reduce the rate limit counters by 1.
//
// When Envoy receives a request that matches the rule, it tries to reduce the
// rate limit counters by the specified number. If the counter doesn't have
// enough capacity, the request is rate limited.
//
// +optional
// +notImplementedHide
Request *RateLimitCostSpecifier `json:"request,omitempty"`
// Response specifies the number to reduce the rate limit counters
// after the response is sent back to the client or the request stream is closed.
//
// The cost is used to reduce the rate limit counters for the matching requests.
// Since the reduction happens after the request stream is complete, the rate limit
// won't be enforced for the current request, but for the subsequent matching requests.
//
// This is optional and if not specified, the rate limit counters are not reduced
// on the response path.
//
// Currently, this is only supported for HTTP Global Rate Limits.
mathetake marked this conversation as resolved.
Show resolved Hide resolved
//
// +optional
// +notImplementedHide
Response *RateLimitCostSpecifier `json:"response,omitempty"`
}

// RateLimitCostSpecifier specifies where the Envoy retrieves the number to reduce the rate limit counters.
//
// +kubebuilder:validation:XValidation:rule="!(has(self.number) && has(self.metadata))",message="only one of number or metadata can be specified"
type RateLimitCostSpecifier struct {
// From specifies where to get the rate limit cost. Currently, only "Number" and "Metadata" are supported.
//
// +kubebuilder:validation:Required
From RateLimitCostFrom `json:"from"`
// Number specifies the fixed usage number to reduce the rate limit counters.
// Using zero can be used to only check the rate limit counters without reducing them.
//
// +optional
// +notImplementedHide
mathetake marked this conversation as resolved.
Show resolved Hide resolved
Number *uint64 `json:"number,omitempty"`
// Metadata specifies the per-request metadata to retrieve the usage number from.
//
// +optional
// +notImplementedHide
mathetake marked this conversation as resolved.
Show resolved Hide resolved
Metadata *RateLimitCostMetadata `json:"metadata,omitempty"`
}

// RateLimitCostFrom specifies the source of the rate limit cost.
// Valid RateLimitCostType values are "Number" and "Metadata".
//
// +kubebuilder:validation:Enum=Number;Metadata
type RateLimitCostFrom string

const (
// RateLimitCostFromNumber specifies the rate limit cost to be a fixed number.
RateLimitCostFromNumber RateLimitCostFrom = "Number"
// RateLimitCostFromMetadata specifies the rate limit cost to be retrieved from the per-request dynamic metadata.
RateLimitCostFromMetadata RateLimitCostFrom = "Metadata"
// TODO: add headers, etc. Anything that can be represented in "Format" can be added here.
// https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage#config-access-log-format
)

// RateLimitCostMetadata specifies the filter metadata to retrieve the usage number from.
type RateLimitCostMetadata struct {
// Namespace is the namespace of the dynamic metadata.
//
// +kubebuilder:validation:Required
Namespace string `json:"namespace"`
// Key is the key to retrieve the usage number from the filter metadata.
//
// +kubebuilder:validation:Required
Key string `json:"key"`
}

// RateLimitSelectCondition specifies the attributes within the traffic flow that can
Expand Down
70 changes: 70 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading