Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

azure-eventhub migration guide: legacy checkpoint sample writes "None" metadata for partitions that were never checkpointed

Abierto Apto para principiantes
  • ¿Algo no está bien?
#49,723 2 comentarios 1 reacción 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@hidingmyname ya está trabajando en esto.

Desde el 29/10/2026.

  • #49759 de @hidingmyname — abierto
  • #49763 de @huong8373tt-beep — abierto

Evaluación

Dificultad
2/5
Tiempo estimado
1-3 horas
Aptitud para principiantes
78/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
azure, python
Área
documentation

Línea de trabajo

Abre sdk/eventhub/azure-eventhub/migration_guide.md y revisa el fragmento de migración de puntos de control heredados. Comprueba cómo trata las entradas a las que les falta offset o sequence_number, y verifica que el prefijo use EVENT_HUB_HOSTNAME, que es la variable definida. La tarea estará terminada cuando el ejemplo omita las entradas que no pueden formar un punto de control y construya el prefijo del blob a partir de la variable definida.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Client customer-reported Event Hubs needs-team-attention question Service Attention
Library name

azure-eventhub (migration_guide.md), azure-eventhub-checkpointstoreblob

Describe the bug

The "migrate checkpoint data from the legacy format" snippet at the end of
sdk/eventhub/azure-eventhub/migration_guide.md converts every blob in the
legacy EventProcessorHost container unconditionally:

    # Translate each legacy checkpoint, storing offset and sequence data into correct
    # blob to align with V5 BlobCheckpointStore.
    for checkpoint in legacy_checkpoints:
        metadata = {
            "offset": str(checkpoint["offset"]),
            "sequencenumber": str(checkpoint["sequence_number"]),
        }
        name = "{}{}".format(prefix, checkpoint["partition_id"])
        container_client.upload_blob(name, data="", metadata=metadata)

The legacy (1.x) EventProcessorHost creates a lease blob for every partition
up front, before anything has been checkpointed. In
azure_storage_checkpoint_manager.py (create_lease_if_not_exists_async) a
fresh AzureBlobLease() is serialized with offset = None and
sequence_number = None, so a partition that was never checkpointed has a blob
like:

{"partition_id": "3", "sequence_number": null, "owner": null, "token": null,
 "epoch": 0, "event_processor_context": null, "offset": null}

json.loads turns those into None, and the snippet writes the metadata
offset="None", sequencenumber="None".

The v5 store then can't read the container. BlobCheckpointStore.list_checkpoints
(azure-eventhub-checkpointstoreblob/.../_blobstoragecs.py, and the same line in
the -aio package) does:

                "offset": str(metadata["offset"]),
                "sequence_number": int(metadata["sequencenumber"]),

int("None") raises ValueError. Because list_checkpoints reads all
partitions in one call, a single unconverted partition makes the whole call
fail. EventProcessor catches the exception in its load-balancing loop, logs
"An error occurred while load-balancing and claiming ownership" and retries,
so the consumer stays up but never starts receiving on any partition it
newly claims.

A second, smaller problem in the same snippet: it defines

EVENT_HUB_HOSTNAME = os.environ["EVENT_HUB_HOSTNAME"]  # <your-eventhub-namespace>.servicebus.windows.net

but builds the blob prefix from a different name that is never defined:

    prefix = "{}/{}/{}/checkpoint/".format(
        EVENT_HUB_FULLY_QUALIFIED_NAMESPACE.lower(),

so as written it fails with NameError before any checkpoint is uploaded.

Why it matters

This snippet is for people moving off the 1.x processor, and having some idle
or never-checkpointed partitions is normal (new partitions, low-traffic
partitions, a consumer group that was only partly used). After running the
migration as documented, the v5 processor connects, logs warnings over and
over, and receives nothing. The error points at load balancing rather than at
the migrated blobs, so it is hard to trace back to the migration step.

Suggested fix

Skip legacy entries that have no checkpoint (a v5 processor will just use its
starting_position for those partitions), and use the variable that is
actually defined:

     prefix = "{}/{}/{}/checkpoint/".format(
-        EVENT_HUB_FULLY_QUALIFIED_NAMESPACE.lower(),
+        EVENT_HUB_HOSTNAME.lower(),
         EVENT_HUB_NAME.lower(),
         EVENT_HUB_CONSUMER_GROUP.lower(),
     )

     # Translate each legacy checkpoint, storing offset and sequence data into correct
     # blob to align with V5 BlobCheckpointStore.
     for checkpoint in legacy_checkpoints:
+        # Partitions that were never checkpointed have no offset/sequence number;
+        # skip them so the V5 processor falls back to its starting position.
+        if checkpoint.get("offset") is None or checkpoint.get("sequence_number") is None:
+            continue
         metadata = {
             "offset": str(checkpoint["offset"]),
             "sequencenumber": str(checkpoint["sequence_number"]),
         }

Both fields are skipped together rather than writing only offset (as .NET
does), because the Python BlobCheckpointStore requires sequencenumber to be
present — metadata["sequencenumber"] would raise KeyError otherwise.

Upstream reference

The .NET migration guide had the same unconditional conversion and fixed it in
Azure/azure-sdk-for-net@2597c2265a ("[Event Hubs] Checkpoint Migration Sample
Tweak", Azure/azure-sdk-for-net#29129, 2022-06-06) — legacy entries with no
offset are skipped when reading, and a missing sequence number is no longer
written as a value:

--- a/sdk/eventhub/Azure.Messaging.EventHubs/MigrationGuide.md
+++ b/sdk/eventhub/Azure.Messaging.EventHubs/MigrationGuide.md
@@ -818,10 +818,14 @@ foreach (var checkpoint in legacyCheckpoints)
 {
     var metadata = new Dictionary<string, string>()
     {
-        { offsetKey, checkpoint.Offset.ToString(CultureInfo.InvariantCulture) },
-        { sequenceKey, checkpoint.SequenceNumber.ToString(CultureInfo.InvariantCulture) }
+        { offsetKey, checkpoint.Offset.ToString(CultureInfo.InvariantCulture) }
     };
 
+    if (checkpoint.SequenceNumber.HasValue)
+    {
+        metadata[sequenceKey] = checkpoint.SequenceNumber.Value.ToString(CultureInfo.InvariantCulture);
+    }
+
     BlobClient blobClient = storageClient.GetBlobClient($"{ prefix }{ checkpoint.PartitionId }");
 
     using var content = new MemoryStream(Array.Empty<byte>());
@@ -857,7 +861,11 @@ private async Task<List<MigrationCheckpoint>> ReadLegacyCheckpoints(
         await (storageClient.GetBlobClient(blobItem.Name)).DownloadToAsync(blobContentStream);
 
         var checkpoint = JsonSerializer.Deserialize<MigrationCheckpoint>(Encoding.UTF8.GetString(blobContentStream.ToArray()));
-        checkpoints.Add(checkpoint);
+
+        if (!string.IsNullOrEmpty(checkpoint.Offset))
+        {
+            checkpoints.Add(checkpoint);
+        }
     }
 
     return checkpoints;

The idea carries over directly — don't turn an empty legacy lease into a
checkpoint. The only Python-specific difference is that the v5 store here
needs both fields, so the entry should be skipped whenever either one is
missing.

I can submit a PR for this.

Lenguaje dominante
Python
Estrellas
5.6k
Forks
3.4k
Merge medio
2 d 16 h
PR fusionados (30 d)
192

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de Azure/azure-sdk-for-python

Todos los issues de Azure/azure-sdk-for-python

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.