nextflow-io/nextflow

Allow AWS S3 Access Points

Open

#1826 aperta il 7 dic 2020

Vedi su GitHub
 (9 commenti) (0 reazioni) (0 assegnatari)Groovy (784 fork)batch import
help wantedstorage/aws

Metriche repository

Star
 (3382 star)
Metriche merge PR
 (Merge medio 16g 9h) (54 PR mergiate in 30 g)

Descrizione

New feature: Allow AWS S3 Access Points

AWS S3 Access Points are unique hostnames attached to an S3 bucket, each with dedicated access policies. This allows large scale access control to be delegated to multiple APs, each dedicated to providing access to one user, rather than combining all access control in one large bucket policy. Larger scale users are increasingly using APs to simplify bucket access control.

Usage scenario

Allow S3 APs to be used as a parameter, for example input files: "--reads", "s3://arn:aws:s3:<region>:<account-id>:accesspoint/<ap-name>/my-fastq-data/*_{1,2}.fastq.gz"

Suggest implementation

The AWS S3 CLI, SDKs, and REST API support Access Points. Currently, using an S3 AP ARN in place of a bucket name results in an error 'The specified bucket is not valid', so it seems the ARN is being used as a literal bucket name. Enabling this feature would involve recognising the AP ARN and using it appropriately in CLI or API calls.

Guida contributor