apache/airflow

Add parameter `deferrable` to `AzureBatchOperator`

已关闭

#59,779 创建于 2025年12月24日

在 GitHub 查看贡献者指南
 (6 条评论) (0 个反应) (0 位负责人)Python (16,781 个派生)batch import
area:async-operatorsgood first issuekind:feature

仓库指标

星标
 (44,809 个星标)
PR 合并指标
 (平均合并 7天 18小时) (30 天内合并 834 个 PR)

描述

Description

The AzureBatchOperator currently has no option to set it to deferrable, which would allow the task to be run by an async low-resource worker ("triggerer"). Therefore Azure Batch tasks block a regular worker, even though they only ping the Azure platform from time to time.

I propose to add this parameter and changing the operator to use asynchronous calls.

As a reference, the AWS Batch operator has a parameter deferrable.

Use case/motivation

Adding this parameter and changing the operator to use asynchronous calls would reduce resource consumption when using Azure Batch.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

贡献者指南

研究方向
Review the current implementation of `AzureBatchOperator` in the Airflow source code, likely located in the `airflow/providers/azure/transfers/` directory. Study the AWS Batch operator's implementation of the `deferrable` parameter and its use of async calls. Examine the triggerer architecture in Airflow documentation. Ensure that adding deferrable support does not break existing synchronous usage.
技术栈
python
领域
backendcloud
议题类型
功能
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
1-2 天
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
活跃
清晰度议题是否清楚说明期望改动、验收标准和下一步。
清晰
前置要求
Python programmingAirflow basics
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
65