apache/airflow

Add parameter `deferrable` to `AzureBatchOperator`

Chiusa

#59.779 aperta il 24 dic 2025

Vedi su GitHubGuida contributor
 (6 commenti) (0 reazioni) (0 assegnatari)Python (16.781 fork)batch import
area:async-operatorsgood first issuekind:feature

Metriche repository

Star
 (44.809 stelle)
Metriche merge PR
 (Merge medio 7g 18h) (834 PR mergiate in 30 g)

Descrizione

Description

The AzureBatchOperator currently has no option to set it to deferrable, which would allow the task to be run by an async low-resource worker ("triggerer"). Therefore Azure Batch tasks block a regular worker, even though they only ping the Azure platform from time to time.

I propose to add this parameter and changing the operator to use asynchronous calls.

As a reference, the AWS Batch operator has a parameter deferrable.

Use case/motivation

Adding this parameter and changing the operator to use asynchronous calls would reduce resource consumption when using Azure Batch.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Guida contributor

Direzione di ricerca
Review the current implementation of `AzureBatchOperator` in the Airflow source code, likely located in the `airflow/providers/azure/transfers/` directory. Study the AWS Batch operator's implementation of the `deferrable` parameter and its use of async calls. Examine the triggerer architecture in Airflow documentation. Ensure that adding deferrable support does not break existing synchronous usage.
Tech stack
python
Dominio
backendcloud
Tipo issue
Funzionalità
DifficoltàQuanto dovrebbe essere impegnativa per un nuovo contributor.
3
Tempo stimatoTempo stimato per completare e verificare uno scope piccolo.
1-2 giorni
Stato attivitàQuanto è probabile che la issue sia ancora attiva e reviewable.
Attiva
ChiarezzaQuanto chiaramente la issue descrive problema, scope e risultato atteso.
Chiara
Prerequisiti
Python programmingAirflow basics
Adatta ai principiantiQuanto la issue è adatta a contributor alla prima esperienza.
65