POC: Sketch out cached filter result API #7513
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alamb
force-pushed
the
alamb/cache_filter_result
branch
from
78f96d1 to
31f2fa1
Compare
alamb
force-pushed
the
alamb/cache_filter_result
branch
from
31f2fa1 to
244e187
Compare
zhuqi-lucas
reviewed
May 16, 2025
| filters: Vec<BooleanArray>, | ||
| } | ||
|
|
||
| impl CachedPredicateResultBuilder { |
There was a problem hiding this comment.
Nice, this is very clear to get the cached result!
alamb
force-pushed
the
alamb/cache_filter_result
branch
2 times, most recently
from
8961196 to
9e91e9f
Compare
zhuqi-lucas
reviewed
May 16, 2025
| /// TODO: potentially incrementally build the result of the predicate | ||
| /// evaluation without holding all the batches in memory. See | ||
| /// <https://github.com/apache/arrow-rs/issues/6692> | ||
| in_progress_arrays: Vec<Box<dyn InProgressArray>>, |
There was a problem hiding this comment.
Thank you @alamb ,
Does it mean, this in_progress_arrays is not the final result for us to generate the final batch?
For example:
Predicate a > 1 => in_progress_array_a filtered by a > 1
Predicate b >2 => in_progress_array_b filtered by b > 2 also based filtered by a > 1, but we don't update the in_progress_array_a
There was a problem hiding this comment.
This is an excellent question
What I was thinking is that CachedPredicateResult would manage the "currently" applied predicate
So in the case where there are multiple predicates, I was thinking of a method like CachedPredicateResult::merge method which could take the result of filtering a and apply the result of filtering by b
We can then put heuristics / logic for if/when we materialize the filters into CachedPredicateResult
But that is sort of speculation at this point -- I don't have it all worked out yet
My plan is to get far enough to show this structure works and can improve performance, and then I'll work on the trickier logic of applying multiple filters
There was a problem hiding this comment.
CachedPredicateResult::merge method which could take the result of filtering a and apply the result of filtering by b
Great idea!
But that is sort of speculation at this point -- I don't have it all worked out yet
Sure, i will continue to review, thank you @alamb !
alamb
force-pushed
the
alamb/cache_filter_result
branch
from
9e91e9f to
147c7a7
Compare
|
I tested this branch using a query that filters and selects the same column (NOTE it is critical to NOT use cargo bench --features="arrow async" --bench arrow_reader_clickbench -- Q24Here are the benchmark results (30ms --> 22ms) (25 % faster) I realize this branch currently uses more memory (to buffer the filter results), but I think the additional memory growth can be limited with a setting. |
alamb
force-pushed
the
alamb/cache_filter_result
branch
from
147c7a7 to
f1f7103
Compare
Amazing result , i think it will be the perfect way instead of page cache, because page caching will have cache missing, but this PR will always cache the result! |
7 tasks
Thanks -- I think one potential problem is that the cached results may consume too much memory (but I will try and handle that shortly) I think we should proceed with starting to merge some refactorings; I left some suggestions here: |
It makes sense! Thank you @alamb. |
This was referenced May 16, 2025
|
🤖 |
|
🤖: Benchmark completed Details
|
It seems regression for Q36/Q37. |
Yes, I agree -- I will figure out why |
alamb
force-pushed
the
alamb/cache_filter_result
branch
from
f1f7103 to
a0e4b29
Compare
I did some profiling: samply record target/release/deps/arrow_reader_clickbench-aef15514767c9665 --bench arrow_reader_clickbench/sync/Q36Basically, the issue is that calling
I added some printlns and it seems like we have 181k rows in total that pass but the number of buffers is crazy (I think this is related to concat not compacting the ByteViewArray). Working on this... |
|
🤖 |
|
🤖: Benchmark completed Details
|
Well, that is looking quite a bit better I am now working on a way to reduce buffering requirements (will require incremental concat'ing) |
alamb
force-pushed
the
alamb/cache_filter_result
branch
from
4dd1d69 to
893819a
Compare
Amazing result @alamb , it looks pretty cool! |
|
🤖 |
Great work, thank you @alamb , i will study and review the details code tomorrow! |
|
🤖: Benchmark completed Details
|
|
🤖 |
|
🤖: Benchmark completed Details
|
Looks like the filter code need some optimization. I will rerun to see if I can repeat it |
|
🤖 |
|
🤖: Benchmark completed Details
|
|
🤖 |
|
🤖: Benchmark completed Details
|
It seems it can be repeated to reproduce. And mostly only happen for filter context But the clickbench benchmark is great. |
Dandandan
reviewed
May 29, 2025
| if let Some((null_count, nulls)) = filter_null_mask(array.nulls(), predicate) { | ||
| builder = builder.null_count(null_count).null_bit_buffer(Some(nulls)); | ||
| } | ||
| let builder = PrimitiveBuilder::<T>::with_capacity(predicate.count); |
There was a problem hiding this comment.
Is it necessary to remove/change those implementations?
I might be pretty hard to match the performance
There was a problem hiding this comment.
No it is not necessary to change the original implementations -- I reworked the existing filter kernels in this PR so that I reuse the existing tests and have confidence in the correctness of the approach.
Dandandan
reviewed
May 29, 2025
arrow-select/src/filter.rs
Outdated
| IterationStrategy::Indices(indices) => { | ||
| append_filtered_nulls(null_buffer_builder, array, predicate); | ||
| let iter = indices.iter().map(|x| values[*x]); | ||
| for v in iter { |
There was a problem hiding this comment.
Good call -- I can find a way to make it faster (direct access to the underlying buffer)
|
My analysis of benchmark results is:
|
alamb
commented
May 29, 2025
| /// 3. `take`-n: a subset of the input array is selected based on the indices provided in a `UInt32Array` or similar. | ||
| /// | ||
| /// This structure handles multiple arrays | ||
| pub struct IncrementalRecordBatchBuilder { |
There was a problem hiding this comment.
I am particularly excited about this structure as I think it is exactly what we need in the DataFusion filter exec
There was a problem hiding this comment.
Great work, very nice to append filter to apply to batch!
|
🤖 |
alamb
added a commit
to alamb/datafusion
that referenced
this pull request
May 29, 2025
|
🤖: Benchmark completed Details
|
|
The benchmarks look better now -- thank you @Dandandan . I will continue to obsess over them after verifying this PR helps with DataFusion |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Draft until:
Which issue does this PR close?
ReadPlanto encapsulate the calculation of what parquet rows to decode #7502Rationale for this change
I am trying to sketch out enough of a cached filter result API to show performance improvements. Once I have done that, I will start proposing how to break it up into smaller PRs
What changes are included in this PR?
Are there any user-facing changes?