Skip to content

Commit f44a73a

Browse files
authored
Xarray-specific filter predicate pushdown. (#113)
Partition pruning: In the Xarray data model, we can always get statistics for the dataset dims, which function like the "primary keys" of the materialized table. These statistics lend themselves to omit processing data that we know we don't need to. For example, ```sql SELECT * FROM air WHERE time > '2020-02-01' ``` Will skip loading partitions whose time ranges are entirely before 2020-02-01.
1 parent 17bc410 commit f44a73a

File tree

6 files changed

+965
-40
lines changed

6 files changed

+965
-40
lines changed

Cargo.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ exclude = [
2020
[dependencies]
2121
arrow = { version = "57.2.0", features = ["pyarrow"] }
2222
async-stream = "0.3"
23+
async-trait = "0.1"
2324
datafusion = { version = "51.0.0" }
2425
datafusion-ffi = { version = "51.0.0" }
2526
futures = { version = "0.3" }

0 commit comments

Comments
 (0)