fix: include ABFS container in object store cache key

[peterxcli]

Which issue does this PR close?

Closes #4993.

Rationale for this change

ABFS URLs encode the container in URL userinfo, for example
abfss://container@account.dfs.core.windows.net/path. The process-wide object store cache
previously keyed stores by scheme, host, and port only, so different containers in the same
storage account shared one cache entry. Since each Azure object store is bound to the container
from its URL, a later read could reuse the first container's store and return data from the wrong
container.

What changes are included in this PR?

• Preserve URL userinfo when building object store keys for abfs and abfss, while retaining
  the existing host-only behavior for other schemes.
• Update the cache documentation to describe container-scoped keys.
• Add an offline regression test that prepares two containers in the same account with identical
  configuration and verifies that they resolve to distinct object store instances.

How are these changes tested?

• cargo test --manifest-path native/Cargo.toml -p datafusion-comet --lib
  (129 passed; 4 ignored)
• cargo clippy --manifest-path native/Cargo.toml -p datafusion-comet --lib --tests -- -D warnings
• cargo fmt --manifest-path native/Cargo.toml --all -- --check

The regression constructs the Azure stores locally and does not issue network requests.

[andygrove]

Nice targeted fix. I confirmed the analysis by reading native/core/src/parquet/objectstore/azure.rs — MicrosoftAzureBuilder::parse_url binds the container into the store instance and the resource Path is container-relative, so a store genuinely cannot be reused across containers. Position::BeforeUsername..AfterPort correctly includes the userinfo when present and collapses to BeforeHost when it isn't, so non-Azure schemes are untouched.

A few small things worth considering:

1. DataFusion's own registry has the same blind spot. RuntimeEnv::register_object_store uses get_url_key, which also strips userinfo (datafusion-execution/src/object_store.rs). So within a single RuntimeEnv, only one abfss store per account can be registered — a second registration would overwrite the first at the DF layer. Today this is fine because Comet constructs a fresh RuntimeEnv per Parquet file read, as the module doc calls out. But that invariant is now load-bearing for correctness on Azure. A sentence tying the two together in the module doc block would help a future reader who tries to share a RuntimeEnv across container reads.

2. A one-line comment on the is_azure_scheme(scheme) branch explaining that ABFS URLs encode the container in the userinfo would save future readers a trip to Object store cache key drops the ABFS container, so two containers in one storage account share a store instance #4993.

3. Two small extensions to the new test, if you want them:

   • Assert that a second call with the same container returns the same Arc, so a future change that accidentally invalidates the same-URL cache path is also caught.
   • Add an s3://bucket@…-style case to pin down that the non-Azure branch still keys by host only.

None of these are blockers.

[peterxcli]

@andygrove thanks for the review, added comment on the abfs branch and new test coverage per your review comment

[peterxcli]

df PR: apache/datafusion#23935

[andygrove]

Fix looks good and I confirmed it works. Two containers in the same account now resolve to their own stores across tasks, where on main the second one gets the first's data.

One question before I approve. Within a single plan, prepare_object_store_with_configs registers against the shared session_ctx.runtime_env() (planner.rs:1528), and DataFusion's get_url_key strips userinfo, so a second container in the same account would overwrite the first in the registry and both scans would resolve to the same store. Your apache/datafusion#23935 fixes that upstream. Do you know whether a single plan can actually reference two containers in one account today? If it can, could we get a follow-up issue tracking the DF bump?

[peterxcli]

prepare_object_store_with_configs registers against the shared session_ctx.runtime_env() (planner.rs:1528), and DataFusion's get_url_key strips userinfo, so a second container in the same account would overwrite the first in the registry and both scans would resolve to the same store. Your apache/datafusion#23935 fixes that upstream. Do you know whether a single plan can actually reference two containers in one account today? If it can, could we get a follow-up issue tracking the DF bump?

We didn't use the datafusion's get_url_key.