Design plan

Components

1. Controller client
   • Use async gRPC calls based on Tonic.
   • Code gen from proto in main repo
2. Controller command line
   • Use structopt
3. Connection factory
4. Connection pool
   • Use ownership model based on a "connectionGuard" to return connections to pool.
5. Decode frames of wire commands
6. Serialization
   • Use ownership to manage borrowing and non-copying.
7. Client Factory
   • Create other components and act as a meta service interface so they can instantiate appropriate dependencies as needed.
   • Need to manage lifetimes.
8. RawClient
   • Need layer to connect receiving bytes to reply listener.
   • Use oneshot channel to manage replies (Like a completable future)
9. Server command line
   • Verify wire compatibility test.
10. ByteStreamWriter
    • Can use append blocks by feeding into a channel and draining it into a single block in a loop.
11. ByteStreamReader
    • Always keep a future of a task pending by pushing completed data into a non-buffered channel.
12. EventSegmentReader
13. EventStreamReaderImpl
    • Pull from segments into a 'current' buffer. (because individual events are handed to calles not blocks)
    • Caller can fill buffer if room from completed futures and then read from buffer. If no completed futures and empty buffer it can select on them.
14. EventStreamWriterImpl
    • Inflight manager - Use enum passed into mpsc on both ends to avoid mutexes and define order.
15. TableSegment
    • Need a read mode to receive updates.
16. TableSynchronizer
    • Ownership + Functions make updates very clean.
17. ReaderGroupState
    • Create MultiKeyLatestItemSequentialProcessor for the non-one-off updates.
18. Authentication
    • Need a client to talk to keycloak to get a token to send to the controller. (This uses a standard protocol)

Unordered

• Metrics
• Logging
• Reactor
• Internal Serialization
• Testing tools