Python: support get partition table with filter

[puchengy]

Feature Request / Improvement

Support get partition tables, especially with filter.

This is especially helpful for airflow sensor job.

Query engine

None

[puchengy]

@Fokko Hi, may I know if we have all the building blocks to implement this feature? Thanks!

[Fokko]

@puchengy Can you provide more context? Would you like to block a partition using an Airflow sensor? I've been thinking about this for a while. For the CDC use case we have a property that we block on: apache/airflow#32786 We could also check if a particular partition exists.

[puchengy]

@puchengy Hi Fokko, so you are saying instead of doing partition compute, we want to store what partitions is produced by a given snapshot in the snapshot summary metadata? If so, how do we deal with snapshot expiration cases (this might also be a question to your Airflow PR as well).

[Fokko]

@puchengy Could you maybe elaborate on the use case in apache/iceberg-python#24? We've moved PyIceberg to a separate repository.