apache/airflow

Add parameter `deferrable` to `AzureBatchOperator`

Open

#59779 opened on Dec 24, 2025

View on GitHub
 (6 comments) (0 reactions) (0 assignees)Python (44,809 stars) (16,781 forks)batch import
area:async-operatorsgood first issuekind:feature

Description

Description

The AzureBatchOperator currently has no option to set it to deferrable, which would allow the task to be run by an async low-resource worker ("triggerer"). Therefore Azure Batch tasks block a regular worker, even though they only ping the Azure platform from time to time.

I propose to add this parameter and changing the operator to use asynchronous calls.

As a reference, the AWS Batch operator has a parameter deferrable.

Use case/motivation

Adding this parameter and changing the operator to use asynchronous calls would reduce resource consumption when using Azure Batch.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor guide

Tech stack
python
Domain
backendcloud
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Python programmingAirflow basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
65
Research direction
Review the current implementation of `AzureBatchOperator` in the Airflow source code, likely located in the `airflow/providers/azure/transfers/` directory. Study the AWS Batch operator's implementation of the `deferrable` parameter and its use of async calls. Examine the triggerer architecture in Airflow documentation. Ensure that adding deferrable support does not break existing synchronous usage.