[pyiceberg_core] Expose IcebergTableProvider to python

[kevinjqliu]

Inspired by #650

I want to expose IcebergTableProvider to Datafusion as python binding using Custom Table Provider

Integration with Python might look something like,

from pyiceberg_core import table_provider
from datafusion import SessionContext

ctx = SessionContext()
iceberg_table_provider = table_provider.create_table_provider(
    metadata_location=metadata_location
)
ctx.register_table_provider("test", iceberg_table_provider)
table = ctx.table("test")
table.show()

[kevinjqliu]

Possibly blocked by apache/datafusion#13851

[kevinjqliu]

Got an example working by building the latest datafusion* libraries locally.
Requires apache/datafusion#13937 and new versions of datafusion* libraries with the PR. Possibly included in the next datafusion release, v45

[Xuanwo]

That's nice! Thank you @kevinjqliu for pushing forward on this.