-
Notifications
You must be signed in to change notification settings - Fork 1k
Closed
Labels
bugparquetChanges to the parquet crateChanges to the parquet crateparquet-variantparquet-variant* cratesparquet-variant* crates
Description
Describe the bug
Currently, variant code is quite willing to support non-variant types such as unsigned integers -- including on the read path for shredded variants in e.g. VariantArray::value
(see the typed_value_to_variant helper function).
I don't know if the shredding spec specifically rejects shredding as non-variant types, but there is a certain logic to the idea. And case 127 of parquet-testing/shredded_variant specifically verifies that the reader rejects a typed_value
column of (parquet) type INTEGER(32, false)
.
To Reproduce
Found while making changes related to shredded struct support; case 127 stopped erroring out with the "expected message ("received empty bytes") and started succeeding.
Expected behavior
The read should fail.
Metadata
Metadata
Assignees
Labels
bugparquetChanges to the parquet crateChanges to the parquet crateparquet-variantparquet-variant* cratesparquet-variant* crates