Commit e08c356
Refactor Velox Writer to Use New Flush Policy Contract (facebookincubator#242)
Summary:
This should be a no-op since no chunking flush policy is currently being used in Prod. but we make three changes in this dif:
1. `writeChunk` now returns a boolean to indicate whether any stream was successfully chunked
2. The previous raw size of the encoded stripe data in the writer context is now stored in the Writer context
3. We update and pass down the memory stats needed by the new flush policy contract
TODO: We will be introducing two more VeloxWriter changes in the next diffs in this stack to:
1. Support per stream chunking instead of always chunking all eligible streams
2. Support breaking down large stream into multiple smaller chunks
Reviewed By: helfman
Differential Revision: D815454331 parent 02c6f98 commit e08c356
File tree
4 files changed
+547
-100
lines changed- dwio/nimble
- common
- velox
- tests
4 files changed
+547
-100
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
74 | 74 | | |
75 | 75 | | |
76 | 76 | | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
77 | 80 | | |
78 | | - | |
79 | | - | |
80 | 81 | | |
81 | 82 | | |
82 | 83 | | |
| |||
0 commit comments