init_transactions docstring says the transaction.timeout.ms clock starts there; it starts at begin_transaction
Los mantenedores suelen responder en 3 días
Evaluación
- Dificultad
- 1/5
- Tiempo estimado
- Menos de una hora
- Aptitud para principiantes
- 91/100
- Tipo de issue
- Documentación
- Claridad
- Bien especificado
- Estado de actividad
- Activo
- Stack tecnológico
- kafka, python
- Área
- documentation
Línea de trabajo
Lee los docstrings de init_transactions y begin_transaction en src/confluent_kafka/src/Producer.c y compara su redacción con la referencia de librdkafka en src/rdkafka.h. Mueve la explicación del tiempo de espera de la transacción a begin_transaction y elimínala de init_transactions; el trabajo estará terminado cuando los docstrings de Python coincidan con el comportamiento documentado del tiempo de espera.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Description
src/confluent_kafka/src/Producer.c, in the init_transactions docstring:
" Upon successful return from this function the application has to\n"
" perform at least one of the following operations within \n"
" `transaction.timeout.ms` to avoid timing out the transaction\n"
" on the broker:\n"
" * produce() (et.al)\n"
" * send_offsets_to_transaction()\n"
" * commit_transaction()\n"
" * abort_transaction()\n"
This belongs on begin_transaction. The transaction timeout applies to an open
transaction, and nothing is open when init_transactions returns — the broker
has no transaction to time out. The clock starts when begin_transaction
succeeds.
librdkafka documents it that way, on rd_kafka_begin_transaction
(src/rdkafka.h):
/**
* @brief Begin a new transaction.
*
* rd_kafka_init_transactions() must have been called successfully (once)
* before this function is called.
*
* Upon successful return from this function the application has to perform at
* least one of the following operations within \c transaction.timeout.ms to
* avoid timing out the transaction on the broker:
* * rd_kafka_produce() (et.al)
* ...
and rd_kafka_init_transactions doesn't mention the timeout. The Python
begin_transaction docstring currently has no mention of it either.
Why it matters
The common pattern is init_transactions() once at startup, then
begin_transaction() per batch. This docstring makes it look like the idle gap
after init is on a clock — so people either rush the first transaction or add
keepalive logic for a timeout that isn't running. And the place where the
timeout genuinely bites, a long-running open transaction, carries no warning.
Suggested fix
Move the paragraph from init_transactions to begin_transaction, matching
librdkafka and the Go client:
" will acquire the internal producer id and epoch, used in all\n"
" future transactional messages issued by this producer instance.\n"
- "\n"
- " Upon successful return from this function the application has to\n"
- " perform at least one of the following operations within \n"
- " `transaction.timeout.ms` to avoid timing out the transaction\n"
- " on the broker:\n"
- " * produce() (et.al)\n"
- " * send_offsets_to_transaction()\n"
- " * commit_transaction()\n"
- " * abort_transaction()\n"
"\n"
" :param float timeout: Maximum time to block in seconds.\n"
" init_transactions() must have been called successfully (once)\n"
" before this function is called.\n"
"\n"
+ " Upon successful return from this function the application has to\n"
+ " perform at least one of the following operations within\n"
+ " `transaction.timeout.ms` to avoid timing out the transaction\n"
+ " on the broker:\n"
+ " * produce() (et.al)\n"
+ " * send_offsets_to_transaction()\n"
+ " * commit_transaction()\n"
+ " * abort_transaction()\n"
+ "\n"
" Any messages produced or offsets sent to a transaction, after\n"
Upstream reference
librdkafka moved this note in a6f81818 ("move note about transaction.timeout.ms
to begin_transaction", 2020-12-09) — removed from
rd_kafka_init_transactions() and added to rd_kafka_begin_transaction():
--- a/src/rdkafka.h
+++ b/src/rdkafka.h
@@ -6951,14 +6951,6 @@
* will acquire the internal producer id and epoch, used in all future
* transactional messages issued by this producer instance.
*
- * Upon successful return from this function the application has to perform at
- * least one of the following operations within \c transaction.timeout.ms to
- * avoid timing out the transaction on the broker:
- * * rd_kafka_produce() (et.al)
- * * rd_kafka_send_offsets_to_transaction()
- * * rd_kafka_commit_transaction()
- * * rd_kafka_abort_transaction()
- *
* @param rk Producer instance.
@@ -7007,6 +6999,14 @@
* rd_kafka_init_transactions() must have been called successfully (once)
* before this function is called.
*
+ * Upon successful return from this function the application has to perform at
+ * least one of the following operations within \c transaction.timeout.ms to
+ * avoid timing out the transaction on the broker:
+ * * rd_kafka_produce() (et.al)
+ * * rd_kafka_send_offsets_to_transaction()
+ * * rd_kafka_commit_transaction()
+ * * rd_kafka_abort_transaction()
+ *
* Any messages produced, offsets sent (rd_kafka_send_offsets_to_transaction()),
confluent-kafka-go applied the same move in confluentinc/confluent-kafka-go@89b6c2c
("Move doc from InitTransactions to BeginTransaction",
confluentinc/confluent-kafka-go#1095, 2024-01-10):
--- a/kafka/producer.go
+++ b/kafka/producer.go
@@ -807,14 +807,6 @@
// will acquire the internal producer id and epoch, used in all future
// transactional messages issued by this producer instance.
//
-// Upon successful return from this function the application has to perform at
-// least one of the following operations within `transaction.timeout.ms` to
-// avoid timing out the transaction on the broker:
-// - `Produce()` (et.al)
-// - `SendOffsetsToTransaction()`
-// - `CommitTransaction()`
-// - `AbortTransaction()`
-//
// Parameters:
@@ -846,6 +838,14 @@
// `InitTransactions()` must have been called successfully (once)
// before this function is called.
//
+// Upon successful return from this function the application has to perform at
+// least one of the following operations within `transaction.timeout.ms` to
+// avoid timing out the transaction on the broker:
+// - `Produce()` (et.al)
+// - `SendOffsetsToTransaction()`
+// - `CommitTransaction()`
+// - `AbortTransaction()`
+//
// Any messages produced, offsets sent (`SendOffsetsToTransaction()`),
So C and Go agree; Python and .NET are the two clients still carrying it in the
old place.
I can submit a PR for this.
- Lenguaje dominante
- Python
- Estrellas
- 528
- Forks
- 977
- Merge medio
- 2 d 19 h
- PR fusionados (30 d)
- 19
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Sin guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de confluentinc/confluent-kafka-python
-
KIP-848 migration guide says never use assign()/unassign() with group.protocol='consumer'; it's only rejected after subscribePosiblemente ocupada @hidingmyname la tomó hace 22 días. Abierto
Dificultad 1/5 1-3 horas Aptitud para principiantes 88/100
confluentinc/confluent-kafka-python#2405 ·
Los mantenedores suelen responder en 3 días
-
Wrong type annotations for read only mapping types.Posiblemente ocupada @RasmusNygren la tomó hace 198 días. Abiertostatus:under-review
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
confluentinc/confluent-kafka-python#2244 ·
Los mantenedores suelen responder en 3 días
-
Wrong signature for `KafkaError`'s 1st parameter in `.pyi`Posiblemente ocupada @wgu9 la tomó hace 159 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
confluentinc/confluent-kafka-python#2243 ·
Los mantenedores suelen responder en 3 días
-
Use joserfc instead of authlib.josePosiblemente ocupada @yvaishalirao la tomó hace 135 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 62/100
confluentinc/confluent-kafka-python#2242 · 2 comentarios ·
Los mantenedores suelen responder en 3 días
-
Producer.poll(timeout) blocks for full timeout after delivery callback fires (regression in 2.13.0)Quizá libre de nuevo Un pull request para esta issue se cerró sin fusionarse. Abiertocomponent:producer
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
confluentinc/confluent-kafka-python#2215 · 1 comentario · 3 reacciones ·
Los mantenedores suelen responder en 3 días
Todos los issues de confluentinc/confluent-kafka-python
Issues similares
-
needs-triage
Dificultad 1/5 Menos de una hora Aptitud para principiantes 88/100
flashinfer-ai/flashinfer#9803 ·
Los mantenedores suelen responder en 1 día
-
bug frontend good first issue help wanted
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
oscarbol09/thesisforge#61 ·
-
api bug performance
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
oscarbol09/audiobard#159 ·
-
area:web p2 product:business triage:active type:bug
Dificultad 1/5 Menos de una hora Aptitud para principiantes 89/100
cuadrao-ai/argus#2328 ·
Los mantenedores suelen responder en 1 día
-
bug frontend good first issue help wanted
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
nikhilb2/simple_invoicing#907 ·