Skip to content

Conversation

Yuhta
Copy link
Contributor

@Yuhta Yuhta commented Oct 7, 2025

Summary:
In case of some irregular encodings of deduplicated arrays or maps, specifically:

  1. The deduplicated field is stored as flat map values, with at least 2 keys;
  2. Array elements in one of the map values are all nulls, and thus stored as null streams;
  3. The reader reads the key with null elements runs first, and the read starts a new run;
  4. Then read the other key, and it is a continuation of previous run, plus some new elements at end.

When the above conditions are all satisfied, the elements vector is changed to a constant vector by NullColumnReader first, then we try to write to it because the second key is reusing the value from previous run, so it tries to copy the cached value into result vector, and throw an exception. Fix this by resetting the result if it is not writable (constant or dictionary).

Differential Revision: D84085512

…eaders

Summary:
In case of some irregular encodings of deduplicated arrays or maps, specifically:

1. The deduplicated field is stored as flat map values, with at least 2 keys;
2. Array elements in one of the map values are all nulls, and thus stored as null streams;
3. The reader reads the key with null elements runs first, and the read starts a new run;
4. Then read the other key, and it is a continuation of previous run, plus some new elements at end.

When the above conditions are all satisfied, the elements vector is changed to a constant vector by `NullColumnReader` first, then we try to write to it because the second key is reusing the value from previous run, so it tries to copy the cached value into result vector, and throw an exception.  Fix this by resetting the result if it is not writable (constant or dictionary).

Differential Revision: D84085512
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 7, 2025
Copy link

meta-codesync bot commented Oct 7, 2025

@Yuhta has exported this pull request. If you are a Meta employee, you can view the originating Diff in D84085512.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot. fb-exported meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant