Is your feature request related to a problem? Please describe.
There is currently no way to detect a corrupted, mid-journal batch that fails to parse. This could lead to nasty cases of silent data loss, where for example acked jobs in a job queue vanish into thin air. Recovering from such corruption isn't possible, since you might end up constructing a logically inconsistent state, so loudly failing in these cases seems to be the only sensible option.
Motivation: I am planning to add raft-based replication to my job queue project sepp and if the failure were loud, I could declare the node dead instead of having it rejoin as a seemingly lagging follower (when in reality it might contain already acked log entries that will now be silently dropped).
Describe the solution you'd like
Quite straight forward, add a new RecoveryMode::Strict (not sure on the name tbh) that would first scan the batches after the corrupted batch for any that are intact (based on the xxh3 hash) and if there are any, emit RecoveryError::MidJournalCorruption.
Describe alternatives you've considered
From the embedding applications perspective nothing can be done, since any raft replicated system assumes durable backing storage or the guarantees are broken.
Additional context
I'd be happy to work on this PR myself.
One wrinkle is that RecoveryError is not marked #[non_exhaustive], so adding a new variant would break exhaustive matches.
Is your feature request related to a problem? Please describe.
There is currently no way to detect a corrupted, mid-journal batch that fails to parse. This could lead to nasty cases of silent data loss, where for example acked jobs in a job queue vanish into thin air. Recovering from such corruption isn't possible, since you might end up constructing a logically inconsistent state, so loudly failing in these cases seems to be the only sensible option.
Motivation: I am planning to add raft-based replication to my job queue project sepp and if the failure were loud, I could declare the node dead instead of having it rejoin as a seemingly lagging follower (when in reality it might contain already acked log entries that will now be silently dropped).
Describe the solution you'd like
Quite straight forward, add a new
RecoveryMode::Strict(not sure on the name tbh) that would first scan the batches after the corrupted batch for any that are intact (based on the xxh3 hash) and if there are any, emitRecoveryError::MidJournalCorruption.Describe alternatives you've considered
From the embedding applications perspective nothing can be done, since any raft replicated system assumes durable backing storage or the guarantees are broken.
Additional context
I'd be happy to work on this PR myself.
One wrinkle is that
RecoveryErroris not marked#[non_exhaustive], so adding a new variant would break exhaustive matches.