Client side load balancing

[jszwedko]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

Creating this issue to roll up other issues related to client-side load balancing into one place.

Use-case is generally to avoid running a separate load balancer to balance requests from Vector sinks to their destinations. See related issues in reference section.

Attempted Solutions

No response

Proposal

No response

References

• Loadbalance data between sinks #18164
• Global Throttling Across Multiple Aggregators #18190
• Vector to vector load balancing & failover #807

Version

vector 0.32.0

[jorgehermo9]

I could give a try to implement this. I was thinking of a loadbalance transform that allows for several strategies; round-robin (this would make the transform stateful), sticky-round-robin (also, stateful transform),weighted round-robin, weighted random, uniform random... What do you think? I think with that transform, this issue and #18164 could be handled with the same solution

[jorgehermo9]

This would imply to have some sort of "duplicated sinks", each one pointing to different endpoints and the loadbalance transform would expose N routes to be consumed by those "duplicated sinks". I see that some sinks have built-in load balancing (#16449 (comment)), but I don't know if it is feasible to implement client-side load balancing at every sink. A loadbalance transform could be used by all sinks out of the box, at the cost of "duplicating the sink" to perform the load balancing. Is this tradeoff okay by your side?

[pront]

Hi @jorgehermo9, the proposed solution basically sounds like a stateful exclusive route transform without user defined conditions, just a strategy. I don't see a problem with the suggested solution. The UX is going to be interesting e.g. how many strategies do you want to support, how does each strategy change the output ports etc.

Generally, I don't see how we can hide this complexity and make it work with existing sinks without changing their UX. Every sink would have to support multiple endpoints and a load balancing strategy. We could have a default strategy but not all sinks support multiple endpoints. I could imagine this co-existing with a load_balance transform.

[jorgehermo9]

Yes, that's exactly what I had in mind @pront! I will come up with a proposal PR and we can iterate UX there if you want :)

We can implement a bunch of strategies, for example, see "Static algorithms" here https://www.cloudflare.com/en-gb/learning/performance/types-of-load-balancing-algorithms/

Dynamic algorithms are also interesting, but I'm not sure if Vector's topology allows us to retrieve the data wee need for that (least connection, response time, etc). I think I would just focus on static ones first..