V3 Tracking issue

[Fokko]

Tracks the progress of the V3 implementation

• New DataTypes
  • UnknownType: Added in feat: Add unknown type #1681
  • Timestamp(Tz)NanoType: Introduced in V3: Introduce timestamp_ns and timestamptz_ns #1632
  • VariantType: Add VariantType #1819
  • Geometry/Geography: Add GeometryType / GeographyType #1820
• Default value:
  • Implement default-value projection #1836
  • Update-schema: Add support for initial-default #1770
  • Prerequisite: Add JSON single value encoding #1805
• Multi-arg transforms
  • Reading of the metadata has been added in Add V3 read support #1554
  • Supporting multi-arg transforms still needs to happen
• Row-level lineage
  • Make PyIceberg aware of row-level lineage #1821
• Deletion vectors
  • Read support is being added in Deletion Vectors read support #1516
  • Write support

[sungwy]

I'd love to get my hands on working with some more new types (when the dependencies are resolved).

Could you assign me to either the VariantType or GeoTypes? (or both? haha) :)

[Fokko]

Hey @sungwy it is first come first serve! :D Please check out the issues of the types itself. I think there are some dependencies, but curious to learn what you think.

[smaheshwar-pltr]

Happy to see an issue tracking this 🚀 !

The V3 spec changes for encryption have merged in apache/iceberg#12162. There's an open PR (edit: now merged) for the resultant changes to table metadata on the Java side here - apache/iceberg#12927.

@Fokko, can we add encryption to the list here, so we're tracking it somewhere for PyIceberg? (Update: I put up #1972 for metadata read change tracking)

[stevie9868]

@Fokko

I am interested in Deletion vectors work if no one is working on it.

Also, I see that read for equality deletes are not implemented yet and write for both positional/equality deletes are also not implemented.

I am aware that the community deprecates position deletes in v3 and might deprecate equality deletes as well in the future so want to ask what is the vision for pyiceberg on these topics, thanks!