FsspecFileIO remote signing (S3V4RestSigner) never fires on modern aiobotocore — S3 requests sent unsigned

pyiceberg/io/fsspec.py の _s3() から始め、S3FileSystem が aiobotocore を通じてクライアントを作成する流れを追跡します。記載されている最新の依存関係バージョンと、S3V4RestSigner を使用する REST カタログで再現してください。実際の PutObject/GetObject リクエストで signer handler が実行され、REST signer エンドポイントが呼び出され、S3 が有効な SigV4 Authorization ヘッダーを受信すれば完了です。

Apache Iceberg version

0.11.1 (latest release), also reproduced on 0.11.0

Please describe the bug 🐞

FsspecFileIO remote request signing (s3.signer=S3V4RestSigner) silently does not sign S3 requests when running against a recent aiobotocore/botocore. The before-sign.s3 handler that _s3() registers never fires for the actual S3 operation, so the request is sent to S3 unsigned and S3 rejects it with:

InvalidRequest: The authorization mechanism you have provided is not supported. Please use Signature Version 4.

This breaks the standard Iceberg REST "remote signing" flow (catalog /v1/config sets s3.signer=S3V4RestSigner, client holds no AWS credentials, each S3 request is signed by the REST signer endpoint) for anyone on a current dependency set.

Root cause

In pyiceberg/io/fsspec.py, _s3() builds the s3fs.S3FileSystem first, then registers the signer on the already-constructed client:

fs = S3FileSystem(**s3_fs_kwargs)
for event_name, event_function in register_events.items():
    fs.s3.meta.events.unregister(event_name, unique_id=1925)
    fs.s3.meta.events.register_last(event_name, event_function, unique_id=1925)

fs.s3 triggers a synchronous connect() that creates one aiobotocore client. But with modern aiobotocore (3.x) the client that actually issues the request is created lazily inside the running event loop (S3FileSystem.set_session), and it does not carry the before-sign.s3 handler registered on the earlier fs.s3 instance. Enabling botocore DEBUG logging confirms the before-sign.s3.PutObject event fires with only the stock handlers (remove_arn_from_signing_path, _set_extra_headers_for_unsigned_request, resolve_s3express_identity) — the S3V4RestSigner handler is absent — and the REST signer endpoint is never called. Because config_kwargs["signature_version"] = UNSIGNED is set, the request goes out with auth_type: none and no Authorization header.

How to reproduce

Environment: pyiceberg[pyiceberg-core]==0.11.1, s3fs==2026.2.0, aiobotocore==3.1.3, botocore==1.42.45, Python 3.13, against any Iceberg REST catalog whose /v1/config returns s3.signer=S3V4RestSigner (i.e. remote signing, no vended credentials).

from pyiceberg.catalog import load_catalog
cat = load_catalog("x", type="rest", uri="<rest-catalog-with-remote-signing>", token="<token>")
t = cat.create_table("ns.t", schema=...)
t.append(some_arrow_table)   # PutObject to S3 goes out UNSIGNED -> InvalidRequest "Please use Signature Version 4"

Pinning botocore < 1.36 (e.g. aiobotocore < 2.16) makes the handler fire again and the signer endpoint gets called — confirming it is a client-lifecycle regression, not a catalog/config problem.

Proposed fix

Register the signer on the aiobotocore session that the filesystem is built from, so every client the session creates inherits the before-sign.s3 handler, instead of registering on the post-construction fs.s3 client. s3fs.S3FileSystem accepts a session= argument:

import aiobotocore.session

session = aiobotocore.session.AioSession()
if signer := properties.get(S3_SIGNER):
    if signer_cls := SIGNERS.get(signer):
        session.register("before-sign.s3", signer_cls(properties))
        config_kwargs["signature_version"] = botocore.UNSIGNED
fs = S3FileSystem(session=session, **s3_fs_kwargs)

Verified locally: with session-level registration the S3V4RestSigner handler fires on the real PutObject/GetObject, the REST signer endpoint is called, and a valid SigV4 Authorization header reaches S3 — on the same modern aiobotocore/botocore that fails today.

Willingness to contribute

I can submit a PR implementing the session-level registration if the maintainers agree with the approach.