MLA - Middleware Layer Abstraction

[heyong4725]

Middleware (communication) layer abstraction (MLA)

We need to implement MLA as a separate crate to provides a middleware abstraction layer that enables scalable, high performance communications for inter async tasks, intra-process (OS threads), interprocess communication on a single computer node or between different nodes in a computer network. MLA needs to support different communication patterns:

• publish-subscribe push / push pattern - the published message is pushed to subscribers
• publish-subscribe push / pull pattern - the published message is write to storage and later pulled by subscribers
• Request / reply pattern
• Point-to-point pattern
• Client / Server pattern

The MLA needs to abstract following details:

• inter-async tasks (e.g., tokio channels), intraprocess (OS threads, e.g., shared memory), interprocess and inter-host / inter-network communication
• different transport layer implementations (shared memory, UDP, TCP)
• builtin support for multiple serialization / deserialization protocols, e.g, capnproto, protobuf, flatbuffers etc
• different language bindings to Rust, Python, C, C++ etc
• telemetry tools for logs, metrics, distributed tracing, live data monitoring (e.g., tap a live data), recording and replay

Rust eco-system has abundant crates to provide underlaying communications, e.g.,:

• tokio / crossbeam provides different types of channels serving different purpose: mpsc, oneshot, broadcast, watch etc
• Tonic provides gRPC services
• Tower provides request/reply service
• Zenoh middleware provides many different pub/sub capabilities

MLA also needs to provide high level APIs:

• publish(topic, value, optional fields):- optional fields may contain senders' identify to help MLA logics to satify above requirements
• subscriber(topic, optional fields)-> future streams
• put(key, value, optional fields)
• get(key, optional fields) -> value
• send(key, msg, optional fields)
• recv(key, optional fields)->value

[phil-opp]

A few unordered thoughts on this:

• For communication across different machines and networks, we need some kind of routing solution. This could be a centralized broker or some kind of decentralized solution as for example used in zenoh. The challenge is that all patterns should use the same routing approach, otherwise some nodes might be reachable using the publish/subscribe API, but not through point-to-point.

  • This requirement makes it much more difficult to support the wide range of communication patterns mentioned above since we cannot easily "pick-and-mix" . So we can either only support the communication systems that implement all the required patterns (not even zenoh does), or we need to manually synchronize the routing state across different frameworks somehow.

• The main advantage of (mpsc) channels as a form of communication within a process is that it enables a higher-level API:

  • Both halves of the channel are within the same process, so each half can trust the other half (thanks to the compiler).
  • This trust makes it possible to send raw data without serialization or verification steps.
  • Since both ends are compiled by the same compiler, the sent data does not even need a stable and specified layout in memory, so almost all types can be used.
  • The compiler can verify that both ends use the same type, so no runtime errors can occur because of mismatched types.
  • No data corruption can happen and no data can be lost since it is never sent over the network.

  By migrating from this API to a more general e.g. publish/subscribe one, we now need to deal with a lot more error cases. The necessary serialization/verification also leads to a considerable performance overhead and drastly reduces the types that can be used with this channel. E.g. sending thread join handles would not be possible anymore, since they might cross a process or even machine boundary.

  So I don't think that we can realisticly replace in-process mpsc channels with a more general middleware.

[heyong4725]

@phil-opp, for your comment#1, definitely we are not looking for "pick-and-mix". My hope is that for each communication patterns, we choose the best middleware provider. For example, for pub/sub pattern, Zenoh will be default since it is by far the best Rust middleware implementation.

[haixuanTao]

When we talk about point-to-point do we strictly need a direct connection. Or, can we just use Zenoh, and create a specific topic and make two points communicate separately on this specific topic ?

Knowing that any client-server communication will require that we initiate a server in addition to the Zenoh session on all nodes.

This server will need additional constraints on IP and Port in order to be seen by all. This can be complex in a k8 environment, for example.

I feel that the best MLA at the moment is just zenoh. If we switch out of zenoh, we can use the tokio ecosystem with a point-to-point approach first.

[heyong4725]

@haixuanTao for point-to-point, I hope it is a direct link, since Zenoh may not be the best efficient way of doing that. We have experienced that performance issue in other project.
For each communication pattern, MLA should transparently select the best underlying technology to implement that, and provide APIs for that.