Rate Limiting Strategies

[theduke]

The official Go client provides rate limiters to prevent spamming the API with requests.

See https://pkg.go.dev/k8s.io/client-go@v0.21.0/util/workqueue

It would be nice to get similar functionality in this project.

Useful would be:

• Global rate limiter for the whole client
• Api<T> specific rate limiter
• Pluggable rate limiting strategies, like in the Go client (exponential backoff, etc)

[MikailBag]

I think kube should somehow reuse existing rate limit implementations from the Tower ecosystem.

In that case, this issue looks like "ability to inject arbitrary Services on client or Api level".

[kazk]

Custom client with layers is now supported. You can do this for global:

use kube::{client::ConfigExt, Client, Config};
use tower::{limit::RateLimitLayer, ServiceBuilder};

let config = Config::infer().await?;
let https = config.native_tls_https_connector()?;
let client = Client::new(
    ServiceBuilder::new()
        .layer(config.base_uri_layer())
        .layer(RateLimitLayer::new(num, duration))
        .service(hyper::Client::builder().build(https))
);

[theduke]

Awesome, thanks!

[clux]

Summary of status after going through issues a bit.

• 1. Global rate limiter strategy above
• 2. Invidual api calls (something that glues backoff with futures - like tokio-retry)
• 3. workqueue based system on the Api type or better ways (by doing multiple manual configs from 1)

For 2; It would be nice to have some examples showcasing individual method retries here. Also plays into #34

Not sure we should do more on the last one here. It would be complicated. This could be improved in the RateLimitLayer in tower by making it configurable based on paths, or by injecting an optional rate limiter deep within Api, or by thinking about if we could clone the Client in such a way that we could customize the layers per copy. (Pretty sure that if we instantiate two instances of a Custom client, you'd be able to get different rate limit layers on each of them. But it's not designed for that). Anyway, investigative work here would be appreciated.

But for now going to limit the scope of this ticket to just get examples that show how to do this:

• example that uses backoff with streams - Add backoff handling for watcher and Controller #703
• example that uses RateLimitLayer in the custom client example - showcase rate limiting strategies on custom clients in example #724