AIP-82good first issuekind:featurekind:metaprovider:amazonprovider:googleprovider:microsoft-azureprovider:redis
仓库指标
- Star
- (44,809 star)
- PR 合并指标
- (平均合并 7天 18小时) (30 天内合并 834 个 PR)
描述
Body
related: #49938
Why
Several users have pointed out that the Event-Driven Scheduling feature introduced in Airflow 3 currently supports only Amazon SQS.
So I add Kafka common message queue provider in #49938, but there are still quite a few common providers that should be useful for airflow users to implement Event-Driven workflows.
What
Here are some common messaging systems:
- Amazon Kinesis
- Amazon MSK
- Amazon SQS
- Apache Kafka
- Azure Message Bus
- Trigger: https://github.com/apache/airflow/pull/53356
- CommonMessageQueue Interface
- Google PubSub https://github.com/apache/airflow/pull/54494
- Redis https://github.com/apache/airflow/pull/52917
reference for implementation: https://github.com/apache/airflow/pull/49938/files
Committer
- I acknowledge that I am a maintainer/committer of the Apache Airflow project.