AIP-82good first issuekind:featurekind:metaprovider:amazonprovider:googleprovider:microsoft-azureprovider:redis
Repository metrics
- Stars
- (44,809 個のスター)
- PR merge metrics
- (PR metrics pending)
説明
Body
related: #49938
Why
Several users have pointed out that the Event-Driven Scheduling feature introduced in Airflow 3 currently supports only Amazon SQS.
So I add Kafka common message queue provider in #49938, but there are still quite a few common providers that should be useful for airflow users to implement Event-Driven workflows.
What
Here are some common messaging systems:
- Amazon Kinesis
- Amazon MSK
- Amazon SQS
- Apache Kafka
- Azure Message Bus
- Trigger: https://github.com/apache/airflow/pull/53356
- CommonMessageQueue Interface
- Google PubSub https://github.com/apache/airflow/pull/54494
- Redis https://github.com/apache/airflow/pull/52917
reference for implementation: https://github.com/apache/airflow/pull/49938/files
Committer
- I acknowledge that I am a maintainer/committer of the Apache Airflow project.