Skip to content

Commit 46c3027

Browse files
macvincentfacebook-github-bot
authored andcommitted
Refactor Velox Writer to Use New Flush Policy Contract (#242)
Summary: This should be a no-op since no chunking flush policy is currently being used in Prod. but we make three changes in this dif: 1. `writeChunk` now returns a boolean to indicate whether any stream was successfully chunked 2. The previous raw size of the encoded stripe data in the writer context is now stored in the Writer context 3. We update and pass down the memory stats needed by the new flush policy contract TODO: We will be introducing two more VeloxWriter changes in the next diffs in this stack to: 1. Support per stream chunking instead of always chunking all eligible streams 2. Support breaking down large stream into multiple smaller chunks Differential Revision: D81545433
1 parent d662f96 commit 46c3027

File tree

4 files changed

+426
-111
lines changed

4 files changed

+426
-111
lines changed

dwio/nimble/common/MetricsLogger.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ struct FileCloseMetrics {
7474
};
7575

7676
enum class LogOperation {
77+
ChunkWrite,
7778
StripeLoad,
7879
StripeFlush,
7980
FileClose,

0 commit comments

Comments
 (0)