init_transactions docstring says the transaction.timeout.ms clock starts there; it starts at begin_transaction
I maintainer di solito rispondono entro 3 giorni
Valutazione
- Difficoltà
- 1/5
- Tempo stimato
- Meno di un'ora
- Idoneità per principianti
- 91/100
- Tipo di issue
- Documentazione
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Attiva
- Stack tecnologico
- kafka, python
- Ambito
- documentation
Direzione di ricerca
Leggi le docstring di init_transactions e begin_transaction in src/confluent_kafka/src/Producer.c e confrontane la formulazione con il riferimento librdkafka in src/rdkafka.h. Sposta la spiegazione del timeout della transazione in begin_transaction e rimuovila da init_transactions; il lavoro è concluso quando le docstring Python corrispondono al comportamento del timeout documentato.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Description
src/confluent_kafka/src/Producer.c, in the init_transactions docstring:
" Upon successful return from this function the application has to\n"
" perform at least one of the following operations within \n"
" `transaction.timeout.ms` to avoid timing out the transaction\n"
" on the broker:\n"
" * produce() (et.al)\n"
" * send_offsets_to_transaction()\n"
" * commit_transaction()\n"
" * abort_transaction()\n"
This belongs on begin_transaction. The transaction timeout applies to an open
transaction, and nothing is open when init_transactions returns — the broker
has no transaction to time out. The clock starts when begin_transaction
succeeds.
librdkafka documents it that way, on rd_kafka_begin_transaction
(src/rdkafka.h):
/**
* @brief Begin a new transaction.
*
* rd_kafka_init_transactions() must have been called successfully (once)
* before this function is called.
*
* Upon successful return from this function the application has to perform at
* least one of the following operations within \c transaction.timeout.ms to
* avoid timing out the transaction on the broker:
* * rd_kafka_produce() (et.al)
* ...
and rd_kafka_init_transactions doesn't mention the timeout. The Python
begin_transaction docstring currently has no mention of it either.
Why it matters
The common pattern is init_transactions() once at startup, then
begin_transaction() per batch. This docstring makes it look like the idle gap
after init is on a clock — so people either rush the first transaction or add
keepalive logic for a timeout that isn't running. And the place where the
timeout genuinely bites, a long-running open transaction, carries no warning.
Suggested fix
Move the paragraph from init_transactions to begin_transaction, matching
librdkafka and the Go client:
" will acquire the internal producer id and epoch, used in all\n"
" future transactional messages issued by this producer instance.\n"
- "\n"
- " Upon successful return from this function the application has to\n"
- " perform at least one of the following operations within \n"
- " `transaction.timeout.ms` to avoid timing out the transaction\n"
- " on the broker:\n"
- " * produce() (et.al)\n"
- " * send_offsets_to_transaction()\n"
- " * commit_transaction()\n"
- " * abort_transaction()\n"
"\n"
" :param float timeout: Maximum time to block in seconds.\n"
" init_transactions() must have been called successfully (once)\n"
" before this function is called.\n"
"\n"
+ " Upon successful return from this function the application has to\n"
+ " perform at least one of the following operations within\n"
+ " `transaction.timeout.ms` to avoid timing out the transaction\n"
+ " on the broker:\n"
+ " * produce() (et.al)\n"
+ " * send_offsets_to_transaction()\n"
+ " * commit_transaction()\n"
+ " * abort_transaction()\n"
+ "\n"
" Any messages produced or offsets sent to a transaction, after\n"
Upstream reference
librdkafka moved this note in a6f81818 ("move note about transaction.timeout.ms
to begin_transaction", 2020-12-09) — removed from
rd_kafka_init_transactions() and added to rd_kafka_begin_transaction():
--- a/src/rdkafka.h
+++ b/src/rdkafka.h
@@ -6951,14 +6951,6 @@
* will acquire the internal producer id and epoch, used in all future
* transactional messages issued by this producer instance.
*
- * Upon successful return from this function the application has to perform at
- * least one of the following operations within \c transaction.timeout.ms to
- * avoid timing out the transaction on the broker:
- * * rd_kafka_produce() (et.al)
- * * rd_kafka_send_offsets_to_transaction()
- * * rd_kafka_commit_transaction()
- * * rd_kafka_abort_transaction()
- *
* @param rk Producer instance.
@@ -7007,6 +6999,14 @@
* rd_kafka_init_transactions() must have been called successfully (once)
* before this function is called.
*
+ * Upon successful return from this function the application has to perform at
+ * least one of the following operations within \c transaction.timeout.ms to
+ * avoid timing out the transaction on the broker:
+ * * rd_kafka_produce() (et.al)
+ * * rd_kafka_send_offsets_to_transaction()
+ * * rd_kafka_commit_transaction()
+ * * rd_kafka_abort_transaction()
+ *
* Any messages produced, offsets sent (rd_kafka_send_offsets_to_transaction()),
confluent-kafka-go applied the same move in confluentinc/confluent-kafka-go@89b6c2c
("Move doc from InitTransactions to BeginTransaction",
confluentinc/confluent-kafka-go#1095, 2024-01-10):
--- a/kafka/producer.go
+++ b/kafka/producer.go
@@ -807,14 +807,6 @@
// will acquire the internal producer id and epoch, used in all future
// transactional messages issued by this producer instance.
//
-// Upon successful return from this function the application has to perform at
-// least one of the following operations within `transaction.timeout.ms` to
-// avoid timing out the transaction on the broker:
-// - `Produce()` (et.al)
-// - `SendOffsetsToTransaction()`
-// - `CommitTransaction()`
-// - `AbortTransaction()`
-//
// Parameters:
@@ -846,6 +838,14 @@
// `InitTransactions()` must have been called successfully (once)
// before this function is called.
//
+// Upon successful return from this function the application has to perform at
+// least one of the following operations within `transaction.timeout.ms` to
+// avoid timing out the transaction on the broker:
+// - `Produce()` (et.al)
+// - `SendOffsetsToTransaction()`
+// - `CommitTransaction()`
+// - `AbortTransaction()`
+//
// Any messages produced, offsets sent (`SendOffsetsToTransaction()`),
So C and Go agree; Python and .NET are the two clients still carrying it in the
old place.
I can submit a PR for this.
- Lingua principale
- Python
- Stelle
- 528
- Fork
- 977
- Merge medio
- 2g 19h
- PR unite (30g)
- 19
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Nessuna guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di confluentinc/confluent-kafka-python
-
KIP-848 migration guide says never use assign()/unassign() with group.protocol='consumer'; it's only rejected after subscribeForse già presa @hidingmyname l’ha presa 22 giorni fa. Aperta
Difficoltà 1/5 1-3 ore Idoneità per principianti 88/100
confluentinc/confluent-kafka-python#2405 ·
I maintainer di solito rispondono entro 3 giorni
-
Wrong type annotations for read only mapping types.Forse già presa @RasmusNygren l’ha presa 198 giorni fa. Apertastatus:under-review
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
confluentinc/confluent-kafka-python#2244 ·
I maintainer di solito rispondono entro 3 giorni
-
Wrong signature for `KafkaError`'s 1st parameter in `.pyi`Forse già presa @wgu9 l’ha presa 159 giorni fa. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
confluentinc/confluent-kafka-python#2243 ·
I maintainer di solito rispondono entro 3 giorni
-
Use joserfc instead of authlib.joseForse già presa @yvaishalirao l’ha presa 135 giorni fa. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 62/100
confluentinc/confluent-kafka-python#2242 · 2 commenti ·
I maintainer di solito rispondono entro 3 giorni
-
Producer.poll(timeout) blocks for full timeout after delivery callback fires (regression in 2.13.0)Forse di nuovo libera Una pull request per questa issue è stata chiusa senza essere unita. Apertacomponent:producer
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
confluentinc/confluent-kafka-python#2215 · 1 commento · 3 reazioni ·
I maintainer di solito rispondono entro 3 giorni
Tutte le issue di confluentinc/confluent-kafka-python
Issue simili
-
needs-triage
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 88/100
flashinfer-ai/flashinfer#9803 ·
I maintainer di solito rispondono entro 1 giorno
-
bug frontend good first issue help wanted
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
oscarbol09/thesisforge#61 ·
-
api bug performance
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
oscarbol09/audiobard#159 ·
-
area:web p2 product:business triage:active type:bug
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 89/100
cuadrao-ai/argus#2328 ·
I maintainer di solito rispondono entro 1 giorno
-
bug frontend good first issue help wanted
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
nikhilb2/simple_invoicing#907 ·