[Variant] Improve documentation and make kernels consistent (#8536)

# Which issue does this PR close?

We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax.

- Closes #8531
- Closes #8532

# Rationale for this change

Basically, I was confused about the casting / conversion functions
available so I want to improve the documentation

# What changes are included in this PR?

1. Add documentation, improve other docs
2. Remove `pub` crates in favor of exporting only the functions/ structs
needed

# Are these changes tested?

Yes by CI
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?

# Are there any user-facing changes?

If there are user-facing changes then we may require documentation to be
updated before approving the PR.

If there are any breaking changes to public APIs, please call them out.

---------

Co-authored-by: Copilot <[email protected]>
Co-authored-by: Ryan Johnson <[email protected]>

Author: 3 people

parquet-variant-compute/benches/variant_kernels.rs

@@ -19,8 +19,9 @@ use arrow::array::{Array, ArrayRef, StringArray};
 use arrow::util::test_util::seedable_rng;
 use criterion::{Criterion, criterion_group, criterion_main};
 use parquet_variant::{Variant, VariantBuilder};
-use parquet_variant_compute::variant_get::{GetOptions, variant_get};
-use parquet_variant_compute::{VariantArray, VariantArrayBuilder, json_to_variant};
+use parquet_variant_compute::{
+    GetOptions, VariantArray, VariantArrayBuilder, json_to_variant, variant_get,
+};
 use rand::Rng;
 use rand::SeedableRng;
 use rand::distr::Alphanumeric;

parquet-variant-compute/src/cast_to_variant.rs

@@ -34,7 +34,7 @@ use arrow_schema::ArrowError;
 /// ```
 /// # use arrow::array::{Array, ArrayRef, Int64Array};
 /// # use parquet_variant::Variant;
-/// # use parquet_variant_compute::cast_to_variant::cast_to_variant;
+/// # use parquet_variant_compute::cast_to_variant;
 /// // input is an Int64Array, which will be cast to a VariantArray
 /// let input = Int64Array::from(vec![Some(1), None, Some(3)]);
 /// let result = cast_to_variant(&input).unwrap();

parquet-variant-compute/src/lib.rs

@@ -20,10 +20,14 @@
 //! ## Main APIs
 //! - [`VariantArray`] : Represents an array of `Variant` values.
 //! - [`VariantArrayBuilder`]: For building [`VariantArray`]
-//! - [`json_to_variant`]: Function to convert a batch of JSON strings to a `VariantArray`.
-//! - [`variant_to_json`]: Function to convert a `VariantArray` to a batch of JSON strings.
-//! - [`mod@cast_to_variant`]: Module to cast other Arrow arrays to `VariantArray`.
-//! - [`variant_get`]: Module to get values from a `VariantArray` using a specified [`VariantPath`]
+//!
+//! # Compute Kernels
+//! - [`json_to_variant()`]: Function to convert Arrays of JSON strings to a `VariantArray`.
+//! - [`variant_to_json()`]: Function to convert a `VariantArray` to arrays of JSON strings.
+//! - [`cast_to_variant()`]: Cast Arrow arrays to `VariantArray`.
+//! - [`variant_get()`]: Convert `VariantArray` (or an inner path) to a strongly-typed Arrow array.
+//! - [`shred_variant()`]: Shred a `VariantArray` according to the provided shredding schema
+//! - [`unshred_variant()`]: Unshred a `VariantArray` to pure binary variant.
 //!
 //! ## 🚧 Work In Progress
 //!
@@ -36,15 +40,15 @@
 //! [Variant issue]: https://github.com/apache/arrow-rs/issues/6736
 
 mod arrow_to_variant;
-pub mod cast_to_variant;
+mod cast_to_variant;
 mod from_json;
 mod shred_variant;
 mod to_json;
 mod type_conversion;
 mod unshred_variant;
 mod variant_array;
 mod variant_array_builder;
-pub mod variant_get;
+mod variant_get;
 mod variant_to_arrow;
 
 pub use variant_array::{BorrowedShreddingState, ShreddingState, VariantArray, VariantType};
@@ -56,3 +60,4 @@ pub use shred_variant::shred_variant;
 pub use to_json::variant_to_json;
 pub use type_conversion::CastOptions;
 pub use unshred_variant::unshred_variant;
+pub use variant_get::{GetOptions, variant_get};

parquet/src/variant.rs

@@ -22,14 +22,11 @@
 //! Note: Requires the `variant_experimental` feature of the `parquet` crate to be enabled.
 //!
 //! # Features
-//! * [`Variant`] represents variant value, which can be an object, list, or primitive.
-//! * [`VariantBuilder`] for building `Variant` values.
-//! * [`VariantArray`] for representing a column of Variant values.
-//! * [`json_to_variant`] and [`variant_to_json`] for converting to/from JSON.
-//! * [`cast_to_variant()`] for casting other Arrow arrays to `VariantArray`.
-//! * [`VariantType`] Arrow ExtensionType for Parquet Variant logical type.
-//!   [`variant_get`] to extracting a value by path and functions to convert
-//!   between `Variant` and JSON.
+//! * Representation of [`Variant`], and [`VariantArray`] for working with
+//!   Variant values (see [`parquet_variant`] for more details)
+//! * Kernels for working with arrays of Variant values
+//!   such as conversion between `Variant` and JSON, and shredding/unshredding
+//!   (see [`parquet_variant_compute`] for more details)
 //!
 //! # Example: Writing a Parquet file with Variant column
 //! ```rust