Allow extension libraries to contribute object stores over FFI

Start by reviewing the upstream datafusion-ffi prerequisite, since this repository has no FFI_ObjectStore or object-store module yet. Then inspect crates/core/src/context.rs and the SessionExtensionComponents design; done means an upstream FFI type exists, this project can import it through the capsule hook, and the extension bundle can optionally expose stores keyed by scheme.

Is your feature request related to a problem or challenge?

Split out of #1676, which proposed adding an object_stores field to SessionExtensionComponents so an extension library could declare its object stores alongside its codecs, functions, and providers and have with_extensions install everything in one call. The other fields in that issue are actionable and are being implemented; this one is blocked, so it is being tracked separately rather than holding up the rest.

An extension library that reads from a storage system datafusion-python does not know about has no way to contribute an object store. Every other extension point in the FFI surface — table providers, catalog providers, functions, codecs, query planners, physical optimizer rules — has a __datafusion_*__ capsule getter that lets a separately compiled library hand over an implementation. Object stores have none.

Describe the solution you'd like

Ultimately, a __datafusion_object_store__ capsule getter following the same convention as the rest of the protocol, so a library can export an ObjectStore implementation across the FFI boundary, plus an object_stores field on SessionExtensionComponents keyed by scheme.

This is blocked upstream and cannot be built here first. There is no FFI_ObjectStore in datafusion-ffi — no object-store module exists in the crate at all. Without one there is nothing for a capsule to carry.

The Python-side surface is also closed today. SessionContext.register_object_store takes StorageContexts, a closed enum over five built-in pyclasses (AmazonS3, GoogleCloudStorage, MicrosoftAzure, LocalFileSystem, HTTP), and crates/core/src/context.rs matches all five exhaustively to pull out an Arc<dyn ObjectStore>. There is no RustWrappedPyObjectStore equivalent to the wrappers the other extension points have. So even a Python-native path would need new work, and a third-party Rust cdylib could not participate at all — it would have to import datafusion.object_store and call back into the host to construct one of our own objects.

The dependency order is therefore: an FFI_ObjectStore upstream in datafusion-ffi, then an importer and __datafusion_object_store__ hook here, then optionally the object_stores bundle field.

Describe alternatives you've considered

What works today. A library that wants to ship a configured store returns one of datafusion-python's own objects and lets the caller register it:

ctx.register_object_store("s3://my-bucket", my_library.configured_s3_store())

That is one line and needs no protocol. It covers the case where the library is packaging credentials or endpoint configuration for a store datafusion-python already supports, which is probably the common case.

What it does not cover is a library implementing a genuinely new ObjectStore — an internal blob service, a content-addressed store, a caching layer in front of another store. That case needs the FFI type and has no workaround short of the library vendoring its own DataFusion.

Adding object_stores to SessionExtensionComponents anyway, carrying the five existing StorageContexts variants. Rejected in #1676: it would be the only field in that dataclass carrying nothing foreign, there would be no capsule to validate, and it would bake the format!("{scheme}{derived_host}") key construction into a tuple shape for no benefit over the one-line call above.

Additional context

Follow-up to #1676; see https://github.com/apache/datafusion-python/issues/1676#issuecomment-5680959562 for the analysis this was split out of. Blocked on an upstream datafusion-ffi change — worth raising in apache/datafusion before any work starts here.