apache/airflow

Add ability to interact with OpenSearch in AWS

已关闭

#39,378 创建于 2024年5月2日

在 GitHub 查看贡献者指南
 (5 条评论) (1 个反应) (0 位负责人)Python (16,781 个派生)batch import
area:providersgood first issuekind:featureprovider:amazon

仓库指标

星标
 (44,809 个星标)
PR 合并指标
 (平均合并 7天 18小时) (30 天内合并 834 个 PR)

描述

Body

We have OpenSearch provider but currently it can't be used with AWS auth mechanisms. We need to create a version of it in AWS provider (overriding only that particular portion of the Hook)

This effort started in https://github.com/apache/airflow/pull/34693

Note this is about querying open search - it's not about submitting Airflow logs to open search (for that we have another task https://github.com/apache/airflow/issues/33619 )

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.

贡献者指南

研究方向
Review the existing OpenSearch provider in Airflow (likely under `airflow/providers/amazon/aws/hooks/opensearch.py`). Understand the current Hook implementation and how it handles authentication. Then examine the linked PR #34693 for prior work and specific changes needed. Create a new provider version under the AWS provider that inherits from the base OpenSearch hook and overrides the authentication method to support AWS IAM, possibly using `requests aws4auth` or boto3. Ensure compatibility with the existing tests and consider adding new tests for AWS auth.
技术栈
pythonaws
领域
backendapicloudsearch
议题类型
功能
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
1-2 天
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
新近可参与
清晰度议题是否清楚说明期望改动、验收标准和下一步。
基本清晰
前置要求
PythonAirflow provider hooksAWS authenticationOpenSearch
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
40