Complete OAuth2 token refresh and expiration support (Java-compatible)

Feature Description

Complete the remaining OAuth2 token refresh and expiration behavior in the REST catalog, using the Java implementation as the compatibility reference. Catalog and child sessions should refresh through the appropriate OAuth2 flow while preserving the session's authentication identity.

The shared auth refresh executor and child-session cache are already implemented by #1033 and #985. This issue builds on those foundations.

Current behavior as of 9c478f75:

• Client-credential catalog sessions support background refresh.
• Credential-based contextual sessions support background refresh when refresh is enabled and an auth executor is provided.
• Contextual and table child sessions are cached, with idle eviction and refresh cleanup.
• Initial RFC 8693 token exchange is supported.

Remaining gaps:

• Background refresh always uses the client credentials flow. token-exchange-enabled is declared but ignored.
• Direct bearer tokens in catalog and child sessions do not refresh automatically.
• Token-exchange child sessions are created with refresh disabled.
• token-expires-in-ms is unsupported. OAuth2 contextual sessions ignore context.properties; table-session bearer-token lifetime properties are also not consumed.
• Java's token-exchange refresh and expired-token recovery behavior are not implemented.

These gaps matter for long-lived contexts and tables: reloading a session from the cache does not update a session already held by an existing caller.

User Experience

Use the existing REST auth APIs and caller-provided auth executor, with the same property names and refresh-flow semantics as Java:

Property	Intended behavior
token-refresh-enabled	Controls automatic refresh for supported catalog and child-session paths.
token-exchange-enabled	Selects token exchange for refresh when enabled; uses the available client credential when disabled, following Java. This setting does not disable the initial exchange of a typed child token.
token-expires-in-ms	Supplies the lifetime used to schedule refresh for a direct bearer token when JWT expiration is unavailable, following Java's property-presence and precedence rules.
auth.session-timeout-ms	Continues to control cache idle eviction independently of token expiration.

For example, an opaque contextual bearer token can supply its lifetime separately from the sensitive token:

iceberg::SessionContext context{
    .session_id = "caller-session",
    .credentials = {{"token", "opaque-access-token"}},
    .properties = {{"token-expires-in-ms", "60000"}},
};

With refresh enabled, an auth executor configured, and a token endpoint that supports the selected flow, the derived session can refresh before expiration. The corresponding direct-bearer table-session path should also consume its lifetime property.

Spec and Compatibility

• Use Java OAuth2Manager, OAuth2Util.AuthSession, and AuthConfig as the behavior reference, together with RFC 8693 for token exchange.
• Refresh the current session token with its current issued token type. Do not enable child refresh by reusing a parent credential in a way that changes the child's authentication identity.
• Match Java's handling of expired tokens and credential-backed recovery, including the required request authentication. Recovery is not an unconditional switch to the catalog identity.
• Consume the specific contextual/table properties Java uses for bearer-token expiration. This does not imply merging all contextual properties into the parent OAuth configuration.
• Preserve request-start-based expiration accounting, existing retry handling, and the caller-provided executor model.
• Preserve cache eviction and shutdown guarantees: removed sessions stop scheduling refreshes, and close/eviction races do not introduce deadlocks or duplicate cleanup.
• Define and test behavior when no auth executor, refresh credential, or usable expiration is available. Do not silently fall back to a different identity.
• No Iceberg storage-format change is required. Any public API additions should be identified in the implementation PRs.

Task Breakdown

Proposed as three implementation PRs. Each PR includes its focused tests; the last step adds cross-path integration coverage and documentation.

• 1. Implement the refresh strategy and expired-token recovery. Honor token-exchange-enabled; exchange the current token and issued token type; retain the Java-compatible client-credential path and request authentication. Cover both setting values, expired-token recovery, retries, and unavailable credentials.
• 2. Support direct bearer expiration and refresh. Add token-expires-in-ms, align JWT/explicit-lifetime precedence and property-presence behavior with Java, consume contextual/table bearer lifetime properties, and enable the catalog/context/table direct-bearer refresh paths. Cover opaque tokens, JWTs, already-expired tokens, and refresh-disabled/no-executor behavior. Depends on step 1.
• 3. Enable token-exchange child refresh and validate the lifecycle. Enable refresh for exchanged contextual/table sessions after the strategy is available. Verify identity-preserving refresh, refreshed expiration/type propagation, cache reuse, failure recovery, and close/eviction during refresh. Document the supported flows and remaining limitations. Depends on step 1; end-to-end validation also covers step 2.

Scope is limited to OAuth2 refresh and expiration parity. SigV4 provider extensibility, TLS configuration, general HTTP retry/idempotency, and storage credential refresh should be tracked separately.

I plan to work on this as a series of small PRs. Feedback on the scope and proposed split is welcome.

Related Issues and References

• #273 — Implement Apache Iceberg REST Catalog Specification.
• #985 — AuthSessionCache for OAuth2 child sessions (merged foundation).
• #1033 — Refresh OAuth2 tokens on a caller-provided executor (merged foundation).
• Java OAuth2Manager.
• Java OAuth2Util.
• Java AuthConfig.
• RFC 8693 — OAuth 2.0 Token Exchange.