Use Polars to avoid data conversion from arrow to pandas

[Yicong-Huang]

Currently our Tuple and Table in Python are implemented as pandas series/datafram, respectively. And we need to convert from arrow data to pandas, for processing UDF, then convert the results back to arrow to send back to jvm.

Polars is a data frame API support arrow as the data format (columnar). I think we could switch to Polars data frame (possibly as a drop in replace of pandas data frame), for the two benefits:

1. to avoid the two extra data conversions.
2. to reduce processing speed, as Polars is written in Rust and benchmark shows its processing speed is faster than pandas.

[chenlica]

Thanks for the suggestion. Do you think it's easy to adopt Polars?

[Yicong-Huang]

Well that part is to figure out. We can give it a try? For instance, we can try with table api and just switch the implementation of table using polars data frame instead of pandas data frame, and compare benchmark.

[Yicong-Huang]

Compatibility might be an issue though. Not sure if polars can support all pandas data frame APIs.