nextflow-io/nextflow

Allow AWS S3 Access Points

Open

#1,826 创建于 2020年12月7日

在 GitHub 查看
 (9 评论) (0 反应) (0 负责人)Groovy (784 fork)batch import
help wantedstorage/aws

仓库指标

Star
 (3,382 star)
PR 合并指标
 (平均合并 16天 9小时) (30 天内合并 54 个 PR)

描述

New feature: Allow AWS S3 Access Points

AWS S3 Access Points are unique hostnames attached to an S3 bucket, each with dedicated access policies. This allows large scale access control to be delegated to multiple APs, each dedicated to providing access to one user, rather than combining all access control in one large bucket policy. Larger scale users are increasingly using APs to simplify bucket access control.

Usage scenario

Allow S3 APs to be used as a parameter, for example input files: "--reads", "s3://arn:aws:s3:<region>:<account-id>:accesspoint/<ap-name>/my-fastq-data/*_{1,2}.fastq.gz"

Suggest implementation

The AWS S3 CLI, SDKs, and REST API support Access Points. Currently, using an S3 AP ARN in place of a bucket name results in an error 'The specified bucket is not valid', so it seems the ARN is being used as a literal bucket name. Enabling this feature would involve recognising the AP ARN and using it appropriately in CLI or API calls.

贡献者指南