[WIP] Backport: Convert escaping Throwable in copyLogSegmentData (0.0.1-SNAPSHOT shape) - #835
Draft
bingkunyangvungle wants to merge 1 commit into
Draft
Conversation
…ption Backport of upstream PR Aiven-Open#833 fix, adapted to the 0.0.1-SNAPSHOT plugin shape where RemoteStorageManager directly performs uploadSegmentLog/uploadIndexes/ uploadManifest (no KafkaRemoteStorageManager delegate yet). Why: an Error (e.g. OutOfMemoryError) escaping copyLogSegmentData bypasses Kafka's RLMTask catch (Exception) and trips ScheduledThreadPoolExecutor.scheduleWithFixedDelay's silent-suppression rule, permanently stopping further copy attempts for that partition. How: outer catch (Throwable) wraps any escaping Throwable as RemoteStorageException, which the broker's existing catch handles, so the schedule survives.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft / WIP — not for merge as-is.
This PR carries the same fix as #833 (Convert escaping Throwable in
copyLogSegmentDatatoRemoteStorageException) but applied against the 0.0.1-SNAPSHOT method shape that exists at tag2024-10-23-1729694047— the version we currently run in our QA Kafka brokers.Why two PRs
#833targets currentmain, whereRemoteStorageManager.copyLogSegmentDatawas refactored to delegate to aKafkaRemoteStorageManager. Cherry-picking#833onto the older 0.0.1-SNAPSHOT branch produces a merge conflict because the method body is structurally different.catch (Throwable)→throw new RemoteStorageException(t)) directly inside the un-delegated upload chain.Why the diff looks like a revert of 284 commits
This PR's base branch is the tag
2024-10-23-1729694047. Compared to currentmain, it's missing the ~284 commits that have landed since (including PR #797 and assorted dependabot bumps). It is not actually meant to merge intomain—mainalready has all those commits. The diff is shown as deletions because git compares tomain.What you actually want to review
Just the single new commit on top of the tag:
The semantic change is identical to the catch-
Throwableblock already proposed in #833.Intended use
Build artifact for our QA clusters that match
0.0.1-SNAPSHOTexactly (same dependency versions as the 2024-10-23 release). The upstream fix that actually goes intomainis #833.Status
WIP — keeping this open for internal review only. Will close once #833 lands and we cut a release built from a
main-based artifact.Refs #820, #833