-
Notifications
You must be signed in to change notification settings - Fork 928
Parquet: Expose accessors from ArrowReaderOptions
#7400
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parquet: Expose accessors from ArrowReaderOptions
#7400
Conversation
ArrowReaderOptions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Adding accessors should not be a breaking change, right?
Thanks @kylebarron @etseidl @mbrobbel |
Thank you @kylebarron for this. I just encountered the same issue and am very happy to see it has already been fixed. CC @alamb: Would it be possible to release a patch that includes this change? This issue is breaking our existing Parquet file reads, as we can't obtain the correct |
We have a release scheduled for later this month: Would you like to make one sooner? If so we could release a 55.0.1 release potentially, or accelerate the timeline for 55.1.0? |
Yeah, I wish we can have this commit sooner so that we can catch up the next release of iceberg-rust. That's will be very appreciated.
Both looks good to me. I didn't check the change list yet. Maybe it's better to cherry pick this commit only? I will leave this decision to you. I'm willing to take tasks needed for this accelerated release. |
I think we have flexibility with release timelines, so I think we should just accelerate the release of 55.1.0. I explained this on #7393 (comment)
Thank you -- since this will just be a normal release I don't think any extra tasks are required. I'll try and get the RC ready for voting today or tomorrow and thus a release out early next week |
Which issue does this PR close?
Ref #7342 (comment)
We can create a standalone issue if needed.
Closes #.
Rationale for this change
I was just updating some of my code to use the v55 release candidate, but it doesn't seem possible for third party code to use the
options
argument inAsyncFileReader::get_metadata
because no internal data fromArrowReaderOptions
is publicly accessible.The internal implementation here on
AsyncRead
andAsyncSeek
accesses thepage_index
andfile_decryption_properties
attributes:arrow-rs/parquet/src/arrow/async_reader/mod.rs
Line 175 in 9322547
arrow-rs/parquet/src/arrow/async_reader/mod.rs
Line 179 in 9322547
But third party code can't do that.
Exposing accessors on
ArrowReaderOptions
allows third party implementations ofAsyncFileReader::get_metadata
to use the values set onArrowReaderOptions
.What changes are included in this PR?
Expose attributes from
ArrowReaderOptions
.Are there any user-facing changes?
New methods.