Replies: 1 comment
-
Hi @Mydurian, ParquetSharp doesn't depend on the names of schema nodes matching the specification, following the backward-compatibility rules, but we currently only support lists that are represented with the three level structure and don't handle data that is missing the inner element layer. I don't think there's a simple way to work around this, but we'd be open to accepting a pull request to add backwards-compatibility support for reading files that don't use the standard schema structure. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi Team,
![image](https://private-user-images.githubusercontent.com/48521631/311628622-6132d709-a399-47a7-b825-4203ca47ab1e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzk1NzExMzgsIm5iZiI6MTczOTU3MDgzOCwicGF0aCI6Ii80ODUyMTYzMS8zMTE2Mjg2MjItNjEzMmQ3MDktYTM5OS00N2E3LWI4MjUtNDIwM2NhNDdhYjFlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjE0VDIyMDcxOFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWU5YmNkNzU4NGMxMTA3ZjA1NDhlZDM4NWU0NjNhMzlhMjA1MTE2ZGY1ZjMxNjcxMTU0N2JmMDA5YmQ5NGNmZWQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.IcjzSFPNpJgyYD7Qe0L9GhXnTtefS2sj7DFJRwgznU8)
When referring the Apache official doc for complex logical type.. I see that the structure definition is somewhere different. Could you help elaborate a little bit more on how to handle some backward-compatibility in ParquetSharp? Thanks in advance.
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md
Beta Was this translation helpful? Give feedback.
All reactions