feat(puffin): Add Puffin crate and CompressionCodec

[fqaiser94]

Part of #744

Summary

• Add Puffin crate
• Add CompressionCodec (see iceberg/puffin-spec/compression-codecs for more information)

Context

• This is the first of a number of PRs to add support for Iceberg Puffin file format.
• It might be helpful to refer to the overarching PR from which these changes were split to understand better how these changes will fit in to the larger picture.

Out of Scope

• Support for LZ4 compression/decompression
  • The Java library does not currently support this either. As the Java implementations are the reference implementations in the Iceberg ecosystem, I would like implement LZ4 support on the Java side first before implementing on the Rust side. At that point, we can also easily check the Java and Rust generated outputs are bit-wise identical.

[Fokko]

This makes sense to me :) Thanks for working on this @fqaiser94 🙌

cc @liurenjie1024 @Xuanwo

[Xuanwo]

Thank you @fqaiser94 for working on this, and also thank you @Fokko for the review. Looks like a good start point for me. Let's move!

[liurenjie1024]

Thanks @fqaiser94 for this pr, just left a comment, others LGTM!

[liurenjie1024]

Do we really need to put this outside of iceberg crate? I'm thinking how will we deal with PuffinReader/PuffindWriter, which will depend on FileIO?

[fqaiser94]

I'm thinking how will we deal with PuffinReader/PuffindWriter, which will depend on FileIO?

I don't understand why this would cause issues since FileIO is a publicly exposed interface? 🤔

Do we really need to put this outside of iceberg crate?

That said, I think you're right.
I don't see any compelling reasons to separate this functionality into a separate crate outside of the iceberg crate.
I didn't consider this option; I just forgot 😅
Here's a quick follow-up PR to move this functionality inside of the existing iceberg crate: #789