refactor: return Result from fallible catalog and schema methods

[gratus00]

Which issue does this PR close?

• Closes Add Result return types to CatalogProvider / SchemaProvider fallible methods #21778.

Rationale for this change

Several CatalogProvider and SchemaProvider methods are fallible in practice, but their signatures do not currently return Result. That forces implementations and callers to either hide errors or treat fallible operations as infallible.

This change makes those fallible paths explicit so errors can be propagated correctly through the catalog API and its downstream callers.

What changes are included in this PR?

This PR updates the following methods to return Result:

• CatalogProvider::schema_names
• CatalogProvider::schema
• SchemaProvider::table_names
• SchemaProvider::table_exist

It also propagates those signature changes through the relevant implementations and call sites, including:

• in-memory and listing catalog/schema providers
• async catalog/schema wrappers
• information schema and dynamic file catalog code
• session/context catalog access paths
• FFI catalog/schema wrappers
• sqllogictest and example code
• affected tests

Are these changes tested?

Yes.

The following commands were run locally:

cargo fmt --all
TMPDIR=/tmp cargo check -p datafusion-catalog -p datafusion-ffi -p datafusion-cli -p datafusion-examples -p datafusion-sqllogictest -p datafusion
TMPDIR=/tmp cargo test -p datafusion-catalog --lib
TMPDIR=/tmp cargo test -p datafusion-ffi --lib
TMPDIR=/tmp cargo test -p datafusion memory_catalog_dereg_nonempty_schema
TMPDIR=/tmp cargo test -p datafusion test_mem_provider
TMPDIR=/tmp cargo check -p datafusion-examples --example udf
TMPDIR=/tmp cargo clippy -p datafusion --lib -- -D warnings

Are there any user-facing changes?

There are no end-user SQL or CLI behavior changes.

This is a public API change for implementers of CatalogProvider and SchemaProvider, so the api change label should be added.

[kosiew]

@gratus00
Thanks for working on this.
I left a couple of small suggestions around error propagation and FFI coverage.

[kosiew]

Nice update here. One small follow-up: this example still turns the newly fallible catalog, schema, and table name lookups into panics via expect.

Since this endpoint already returns Result<_, Status>, it might be worth changing tables to return Result<RecordBatch, Status> and mapping these errors to Status::internal(...) with ?. That would keep the example aligned with the PR's invariant that catalog failures propagate to callers instead of aborting the server.

[kosiew]

The FFI round-trip tests cover the successful path after the ABI changed to FFI_Result, which is great. It would also be helpful to have a small failing provider test that asserts failures from schema_names and schema cross the FFI boundary as DataFusionErrors.

That would give this PR's main contract some coverage at the FFI boundary too.

[gratus00]

See if you like these?

[gratus00]

Hi @kosiew, thank you for your review! I'll take a look at this hopefully next week if there doesn't seem to be a rush!

[kosiew]

@gratus00
I will review this again after resolution of the conflicts.

[kosiew]

@gratus00
Thanks for the work here. I think this needs one upgrade-guide note before it lands, since the public provider trait signatures are changing and downstream implementors will need a clear migration path.

[kosiew]

Small cleanup suggestion: this manual if let Some can be collapsed to the direct lookup result, since the map already stores Arc<dyn SchemaProvider>:

Ok(schemas.get(name).cloned())

[alamb]

I also added the api change to this PR