Repository-Metriken
- Stars
- (44.809 Sterne)
- PR-Merge-Metriken
- (Durchschn. Merge 7T 18h) (834 gemergte PRs in 30 T)
Beschreibung
Description
Standardize similar AWS transfer operators to have similar functionality, as well as similar naming conventions.
This would start with updating transfer operators using S3 as a source or destination to have similar behavior and naming.
Use case/motivation
There seems to be substantial discrepancies between transfer operators, especially those using S3 as a source or destination. For example, in the S3ToSFTPOperator, only a single file can be transferred from the SFTP to the S3 bucket. However, in the FTPToS3Operator an arbitrary number of files can be transferred from S3 to an FTP site. To add to this, the S3ToFTPOperator only supports a single file being transferred.
These sort of discrepancies exist for a number of transfer operators. In addition to that, parameter names are quite different from operator to operator, which makes using them quite difficult.
This will allow for a more seamless experience for Airflow users, and ensure common functionality across similar tools.
Related issues
No related issues.
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct