New S3-FileSystem Implementation not compatible with s3connection resource reference

TrinoCatalog の S3 リファレンス処理と S3Connection の統合から始め、fs.native-s3.enabled が true の場合にリファレンスが hive.s3.* プロパティをどのように生成するかを追跡します。Trino 451 で設定を再現し、生成されたプロパティが未使用プロパティのエラーなしで受け入れられることを確認します。

Affected Stackable version

24.11.0

Affected Trino version

451

Current and expected behavior

We are currently migrating from legacy s3 file system to the new standard s3 filesystem (https://trino.io/docs/current/object-storage/file-system-s3.html#migration-from-legacy-s3-file-system)

we encountered an error using TrinoCatalogs and S3Connection as reference like this:

apiVersion: trino.stackable.tech/v1alpha1
kind: TrinoCatalog
metadata:
  name: mycatalog
  namespace: myns
spec:
  configOverrides:
    fs.native-s3.enabled: "true"
    hive.iceberg-catalog-name: mycatalog-iceberg
    hive.metastore.authentication.type: KERBEROS
    hive.metastore.client.keytab: /stackable/config/keytab/keytab
    hive.metastore.client.principal: USER@REALM
    hive.metastore.service.principal: <redacted>
    hive.metastore.thrift.impersonation.enabled: "false"
    hive.partition-projection-enabled: "true"
    s3.endpoint: MYENDPOINT
    s3.max-connections: "40"
    s3.path-style-access: "true"
    s3.region: us-east-1
    s3.retry-mode: STANDARD
    s3.socket-connect-timeout: 300s
    s3.socket-read-timeout: 300s
    s3.streaming.part-size: 100MB
  connector:
    hive:
      metastore:
        configMap: product-mycatalog-hive
      s3:
        reference: trino

here, trino is a s3 connection looking like this:

apiVersion: s3.stackable.tech/v1alpha1
kind: S3Connection
metadata:
  name: trino
  namespace: myns
spec:
  accessStyle: Path
  credentials:
    secretClass: s3-access-trino
  host: myhost
  port: myport
  tls:
    verification:
      server:
        caCert:
          secretClass: ca-cert

this is the error we obtain:

 
1) Error: Configuration property 'hive.s3.aws-access-key' was not used
 
2) Error: Configuration property 'hive.s3.aws-secret-key' was not used
 
3) Error: Configuration property 'hive.s3.endpoint' was not used
 
4) Error: Configuration property 'hive.s3.path-style-access' was not used
 
5) Error: Configuration property 'hive.s3.ssl.enabled' was not used
 
5 errors
io.airlift.bootstrap.ApplicationConfigurationException: Configuration errors:
 
1) Error: Configuration property 'hive.s3.aws-access-key' was not used
 
2) Error: Configuration property 'hive.s3.aws-secret-key' was not used
 
3) Error: Configuration property 'hive.s3.endpoint' was not used
 
4) Error: Configuration property 'hive.s3.path-style-access' was not used
 
5) Error: Configuration property 'hive.s3.ssl.enabled' was not used
 
5 errors
        at io.airlift.bootstrap.Bootstrap.configure(Bootstrap.java:217)
        at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:246)
        at io.trino.plugin.iceberg.IcebergConnectorFactory.createConnector(IcebergConnectorFactory.java:116)
        at io.trino.plugin.iceberg.IcebergConnectorFactory.create(IcebergConnectorFactory.java:78)
        at io.trino.connector.DefaultCatalogFactory.createConnector(DefaultCatalogFactory.java:207)
        at io.trino.connector.DefaultCatalogFactory.createCatalog(DefaultCatalogFactory.java:124)
        at io.trino.connector.LazyCatalogFactory.createCatalog(LazyCatalogFactory.java:45)
        at io.trino.connector.StaticCatalogManager.lambda$loadInitialCatalogs$1(StaticCatalogManager.java:161)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
        at java.base/java.util.concurrent.ExecutorCompletionService.submit(Unknown Source)
        at io.trino.util.Executors.executeUntilFailure(Executors.java:46)
        at io.trino.connector.StaticCatalogManager.loadInitialCatalogs(StaticCatalogManager.java:155)
        at io.trino.server.Server.doStart(Server.java:155)
        at io.trino.server.Server.lambda$start$0(Server.java:93)
        at io.trino.$gen.Trino_451____20250227_152637_1.run(Unknown Source)
        at io.trino.server.Server.start(Server.java:93)
        at io.trino.server.TrinoServer.main(TrinoServer.java:37)

it looks like properties like hive.s3.* are added, but incompatible when fs.native-s3.enabled = true.

Possible solution

No response

Additional context

No response

Environment

No response

Would you like to work on fixing this bug?

None