Migrate microsoft.azure provider to azure-batch 15.x (BatchClient) — unblock <15 cap from #66452

Background

PR #66452 caps azure-batch<15.0.0 in providers/microsoft/azure/pyproject.toml because the azure-batch 15.x release line (15.1.0 GA on PyPI 2026-05-01) is a complete rewrite (Azure SDK "track 2") that removes BatchServiceClient, batch_auth, and the model classes that AzureBatchHook and AzureBatchOperator currently depend on.

The cap is a workaround, not a fix. This issue tracks the work needed to lift it.

What needs to change

1. providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/batch.py

The hook currently imports:

from azure.batch import BatchServiceClient, batch_auth, models as batch_models
from azure.batch.models import JobAddParameter, PoolAddParameter, TaskAddParameter

Track-2 equivalents to migrate to:

Old (≤14.x)	New (15.x)
BatchServiceClient(credentials, batch_url=...)	BatchClient(endpoint=..., credential=...) (sync) — module path TBD per release
batch_auth.SharedKeyCredentials(login, key)	AzureNamedKeyCredential (or whatever shared-key support track 2 ships)
batch_models.PoolAddParameter	BatchPoolCreateContent (typical track-2 rename pattern)
batch_models.JobAddParameter	BatchJobCreateContent
batch_models.TaskAddParameter	BatchTaskCreateContent
batch_models.VirtualMachineConfiguration	renamed in track 2 — verify in changelog
batch_models.CloudServiceConfiguration	removed in 15.x (cloud-service VMs deprecated by Azure) — drop the code path
batch_models.ImageReference	BatchVmImageReference
batch_models.BatchErrorException	azure.core.exceptions.HttpResponseError (track-2 unified exceptions)
batch_models.AccountListSupportedImagesOptions	replaced with kwargs on the list method
batch_models.ComputeNodeState	enum renamed — check 15.x docs
batch_client.pool.list() etc.	sync client methods may rename to list_pools() etc. (track-2 convention)

2. providers/microsoft/azure/src/airflow/providers/microsoft/azure/operators/batch.py

AzureBatchOperator's public constructor exposes 8 model types as parameter type hints:

batch_job_manager_task: batch_models.JobManagerTask | None = None,
batch_job_preparation_task: batch_models.JobPreparationTask | None = None,
batch_job_release_task: batch_models.JobReleaseTask | None = None,
batch_task_container_settings: batch_models.TaskContainerSettings | None = None,
batch_start_task: batch_models.StartTask | None = None,
batch_task_resource_files: list[batch_models.ResourceFile] | None = None,
batch_task_output_files: list[batch_models.OutputFile] | None = None,
batch_task_user_identity: batch_models.UserIdentity | None = None,

These are part of the operator's public API — renaming the types is a breaking change for any DAG that imports them. Migration needs:

• A deprecation cycle: keep old names re-exported as aliases for one provider release with a DeprecationWarning, then remove.
• Or a major-version bump of the microsoft.azure provider.

3. Tests

• providers/microsoft/azure/tests/unit/microsoft/azure/hooks/test_batch.py
• providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_batch.py

These mock the old SDK's class names. They'll need to be re-mocked against the new client surface.

4. Docs

• providers/microsoft/azure/docs/operators/batch.rst (if it exists) — example snippets reference old model names.
• providers/microsoft/azure/docs/index.rst — bump the azure-batch requirement back to a single lower bound once the cap is removed.

5. Authentication paths

The current hook supports two auth modes:

• Shared-key (batch_auth.SharedKeyCredentials)
• Azure Identity (AzureIdentityCredentialAdapter)

Both need re-validation against the 15.x client. The AzureIdentityCredentialAdapter shim in airflow.providers.microsoft.azure.hooks._azure_credential may be obsolete in track 2 (the new client accepts azure-identity credentials natively).

Acceptance criteria

The cap can be removed (azure-batch>=15.0.0 allowed) when all of the following hold:

• AzureBatchHook works against azure-batch >= 15.0.0 with both shared-key and identity credentials.
• AzureBatchOperator accepts the new model types.
• providers/microsoft/azure/tests/unit/microsoft/azure/{hooks,operators}/test_batch.py pass against the new SDK.
• Compat tests (Compat 2.11.1:P3.10, Compat 3.0.6:P3.10, Compat 3.1.8:P3.10, Compat 3.2.1:P3.10) pass — these run the provider's tests against past Airflow versions, and the migration must keep import-time compatibility with all of them.
• The deprecation strategy for the renamed model-type hints in AzureBatchOperator's public signature is decided and documented (one-release alias warnings vs. straight provider major bump).
• Changelog entry in providers/microsoft/azure/docs/changelog.rst describing the upgrade and any breaking surface.
• The <15.0.0 cap is removed from providers/microsoft/azure/pyproject.toml and the comment pointing to this issue is removed.
• The matching cap in providers/microsoft/azure/docs/index.rst is removed.

References

• PR #66452 — applied the cap.
• azure-batch CHANGELOG — upstream notes on the rewrite.
• Azure SDK track-2 design guidelines — naming conventions for client + model renames.

Notes on the cap location

When this issue is opened, please update the workaround comment in providers/microsoft/azure/pyproject.toml (currently says "Lifting the upper bound cap needs a full hook rewrite") to also point at this issue's URL, per the project's convention for tracked workarounds.