ADR-57: JetStream durable stream sourcing/mirroring #389
Conversation
Signed-off-by: Maurice van Veen <[email protected]>
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
|
@roeschter Please add comments inline using the review function, otherwise it's extremely difficult to thread and track replies to individual comments. |
roeschter
left a comment
roeschter
left a comment
There was a problem hiding this comment.
Missing specifications?
-Will the sourcing side verify that the consumer has the right AckPolicy of AckFlowControl?
-What will happen to sourcing from a workqueue is NOT consumer is set? WE shouldn't fail (compatibility), but should we at least print a warning
adr/ADR-57.md
Outdated
|
|
||
| Some additional tooling will be required to create the durable consumer with the proper configuration. But through the | ||
| use of new fields/values on the consumer configuration, the server will be able to help enforce the correct | ||
| configuration. |
There was a problem hiding this comment.
This is too vague. Which fields? What will be enforced? We need to at least reference where the details are specified,
There was a problem hiding this comment.
Updated to:
... But through the use of a new AckPolicy=AckFlowControl field, the server will be able to help enforce the correct configuration.
The enforcing it does is then mentioned in the "Consumer configuration" section, specifically:
- Requires
FlowControlandHeartbeatto be set.
adr/ADR-57.md
Outdated
|
|
||
| The durable consumer used for stream sourcing/mirroring will need to be just as performant as the current ephemeral | ||
| variant. The current ephemeral consumer configuration uses `AckNone` which is problematic for WorkQueue and Interest | ||
| streams. A different `AckPolicy` will need to be used to ensure that messages are not lost. |
There was a problem hiding this comment.
A new AckPolicy=AckFlowControl will be introduce.
There was a problem hiding this comment.
This is just introductory text explaining AckNone can't be used, the mention of the new field is done below:
- Uses
AckPolicy=AckFlowControlinstead ofAckNone
adr/ADR-57.md
Outdated
| - `AckPolicy=AckFlowControl` will function like `AckAll` although the server will not use the ack reply on the received | ||
| messages. | ||
| - The server responds to the flow control messages, including the stream sequence (`Nats-Last-Stream`) and delivery | ||
| sequence (`Nats-Last-Consumer`) as headers to specify which messages have been successfully stored. |
There was a problem hiding this comment.
The receiving stream responds with a flow control message, which includes the stream sequence (Nats-Last-Stream) and delivery sequence (Nats-Last-Consumer) as headers to signalling which messages have been successfully stored
There was a problem hiding this comment.
Slightly updated to:
The receiving server responds to the flow control messages, which includes the stream sequence (Nats-Last-Stream) and delivery sequence (Nats-Last-Consumer) as headers to signal which messages have been successfully stored.
Specifically the server that's receiving the flow control message responds to it. It doesn't send flow control messages itself.
adr/ADR-57.md
Outdated
| messages. | ||
| - The server responds to the flow control messages, including the stream sequence (`Nats-Last-Stream`) and delivery | ||
| sequence (`Nats-Last-Consumer`) as headers to specify which messages have been successfully stored. | ||
| - The server receiving the flow control response will ack messages based on these stream/delivery sequences. |
There was a problem hiding this comment.
The server receiving the flow control response will internally acknowledge all prior messages in the consumer according to its retention policy. For work queue and interest based streams this may result in messages deletion.
There was a problem hiding this comment.
Have kept the addition of For WorkQueue and Interest streams this may result in messages deletion. But am inclined to not change the former. A response to a flow control message will not mean ALL prior messages are to be acknowledged. Only those messages that are below the stream/delivery sequences are acked.
adr/ADR-57.md
Outdated
| - The consumer will be reset, resembling the delivery state of creating a new consumer with `opt_start_seq` set to the | ||
| specified sequence. | ||
| - The pending and redelivered messages will always be reset. | ||
| - The delivered stream/consumer sequences will always be reset. |
There was a problem hiding this comment.
The delivered consumer sequence will always be reset.
There was a problem hiding this comment.
Have updated this to delivered stream and consumer sequence. These are two independent sequences that need to be reset.
Signed-off-by: Maurice van Veen <[email protected]>
Signed-off-by: Maurice van Veen <[email protected]>
|
LGTM |
|
Opened a draft PR on the server with a working version for this. I'll update this ADR with some refinements soon. |
|
This ADR is now also updated to reflect the initial implemention. Requiring to specify |
Signed-off-by: Maurice van Veen <[email protected]>
MauriceVanVeen
force-pushed
the
maurice/durable-sourcing
branch
from
f741e0a to
ed4847e
Compare
Signed-off-by: Maurice van Veen <[email protected]>
MauriceVanVeen
force-pushed
the
maurice/durable-sourcing
branch
from
c381cc7 to
06e8ab3
Compare
ripienaar
left a comment
ripienaar
left a comment
There was a problem hiding this comment.
LGTM, feels like we need a proper ok from @derekcollison also maybe?
Stream sourcing/mirroring on WorkQueue or Interest streams is not reliable due to the use of an ephemeral
AckNoneconsumer. This ADR proposes a design for durable stream sourcing/mirroring that works reliably on all stream retention types.Signed-off-by: Maurice van Veen [email protected]