[core] Supports only execute specified actions during the writing process. #4884
+555
−50
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
LinMingQiang
force-pushed
the
master-feat-addconf-toskip-action
branch
2 times, most recently
from
9d3fcef to
ed49eb8
Compare
wwj6591812
reviewed
Jan 12, 2025
|
Thanks for @LinMingQiang prepare this pr. |
xuzifu666
reviewed
Jan 12, 2025
LinMingQiang
force-pushed
the
master-feat-addconf-toskip-action
branch
from
7250389 to
b3867f8
Compare
|
+1 |
Zouxxyy
reviewed
Jan 13, 2025
LinMingQiang
force-pushed
the
master-feat-addconf-toskip-action
branch
from
b3867f8 to
99dcebd
Compare
LinMingQiang
force-pushed
the
master-feat-addconf-toskip-action
branch
5 times, most recently
from
2d26f9d to
fd50ccd
Compare
LinMingQiang
changed the title
LinMingQiang
changed the title
|
@JingsongLi PTAL, thanks! |
JingsongLi
reviewed
Feb 11, 2025
| TAG_AUTOMATIC_CREATION("tag-automatic-creation"), | ||
|
|
||
| // Actions during writing. | ||
| MINOR_COMPACT("minor-compact"), |
There was a problem hiding this comment.
It is strange here, in streaming mode, it is hard to say just doing minor compact.
There was a problem hiding this comment.
We can only keep full-compact and minor-compact action can be removed. minor-compact keep the original default logic.
JingsongLi
reviewed
Feb 11, 2025
| || doWriteActions.contains(WriteAction.PARTITION_EXPIRE); | ||
| } | ||
|
|
||
| public boolean doSnapshotExpireAction(Set<WriteAction> doWriteActions) { |
There was a problem hiding this comment.
All these methods can be in WriteAction, they are just static methods.
LinMingQiang
force-pushed
the
master-feat-addconf-toskip-action
branch
from
fd50ccd to
3702c60
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
There are several operations during data writing such as
partition-expire,snapshot-expire,create-tag...Currently, these actions are either executed together or not executed at all in the framework.
The purpose of this PR is to supports skipping specified actions during writing.
For example : set
'write.skip-actions'='partition-expire', the partition expiration action will be skipped during data writing, butsnapshot-expire/auto create tag/full compactwill still execute normally.Linked issue: close #xxx
Tests
API and Format
Documentation