init_transactions docstring says the transaction.timeout.ms clock starts there; it starts at begin_transaction
メンテナーはふだん 3 日以内に返信
評価
- 難易度
- 1/5
- 見積もり時間
- 1時間未満
- 初心者へのやさしさ
- 91/100
- issue の種類
- ドキュメント
- 明瞭さ
- 明確に書かれている
- 活発さ
- 活発
- 技術スタック
- kafka, python
調査の方向性
src/confluent_kafka/src/Producer.c の init_transactions と begin_transaction の docstring を読み、src/rdkafka.h にある librdkafka のリファレンスと表現を比較してください。トランザクションのタイムアウトに関する説明を begin_transaction に移し、init_transactions から削除してください。Python の docstring が文書化されたタイムアウトの動作と一致すれば完了です。
索引モデルが issue の本文から書いたものです。
説明
Description
src/confluent_kafka/src/Producer.c, in the init_transactions docstring:
" Upon successful return from this function the application has to\n"
" perform at least one of the following operations within \n"
" `transaction.timeout.ms` to avoid timing out the transaction\n"
" on the broker:\n"
" * produce() (et.al)\n"
" * send_offsets_to_transaction()\n"
" * commit_transaction()\n"
" * abort_transaction()\n"
This belongs on begin_transaction. The transaction timeout applies to an open
transaction, and nothing is open when init_transactions returns — the broker
has no transaction to time out. The clock starts when begin_transaction
succeeds.
librdkafka documents it that way, on rd_kafka_begin_transaction
(src/rdkafka.h):
/**
* @brief Begin a new transaction.
*
* rd_kafka_init_transactions() must have been called successfully (once)
* before this function is called.
*
* Upon successful return from this function the application has to perform at
* least one of the following operations within \c transaction.timeout.ms to
* avoid timing out the transaction on the broker:
* * rd_kafka_produce() (et.al)
* ...
and rd_kafka_init_transactions doesn't mention the timeout. The Python
begin_transaction docstring currently has no mention of it either.
Why it matters
The common pattern is init_transactions() once at startup, then
begin_transaction() per batch. This docstring makes it look like the idle gap
after init is on a clock — so people either rush the first transaction or add
keepalive logic for a timeout that isn't running. And the place where the
timeout genuinely bites, a long-running open transaction, carries no warning.
Suggested fix
Move the paragraph from init_transactions to begin_transaction, matching
librdkafka and the Go client:
" will acquire the internal producer id and epoch, used in all\n"
" future transactional messages issued by this producer instance.\n"
- "\n"
- " Upon successful return from this function the application has to\n"
- " perform at least one of the following operations within \n"
- " `transaction.timeout.ms` to avoid timing out the transaction\n"
- " on the broker:\n"
- " * produce() (et.al)\n"
- " * send_offsets_to_transaction()\n"
- " * commit_transaction()\n"
- " * abort_transaction()\n"
"\n"
" :param float timeout: Maximum time to block in seconds.\n"
" init_transactions() must have been called successfully (once)\n"
" before this function is called.\n"
"\n"
+ " Upon successful return from this function the application has to\n"
+ " perform at least one of the following operations within\n"
+ " `transaction.timeout.ms` to avoid timing out the transaction\n"
+ " on the broker:\n"
+ " * produce() (et.al)\n"
+ " * send_offsets_to_transaction()\n"
+ " * commit_transaction()\n"
+ " * abort_transaction()\n"
+ "\n"
" Any messages produced or offsets sent to a transaction, after\n"
Upstream reference
librdkafka moved this note in a6f81818 ("move note about transaction.timeout.ms
to begin_transaction", 2020-12-09) — removed from
rd_kafka_init_transactions() and added to rd_kafka_begin_transaction():
--- a/src/rdkafka.h
+++ b/src/rdkafka.h
@@ -6951,14 +6951,6 @@
* will acquire the internal producer id and epoch, used in all future
* transactional messages issued by this producer instance.
*
- * Upon successful return from this function the application has to perform at
- * least one of the following operations within \c transaction.timeout.ms to
- * avoid timing out the transaction on the broker:
- * * rd_kafka_produce() (et.al)
- * * rd_kafka_send_offsets_to_transaction()
- * * rd_kafka_commit_transaction()
- * * rd_kafka_abort_transaction()
- *
* @param rk Producer instance.
@@ -7007,6 +6999,14 @@
* rd_kafka_init_transactions() must have been called successfully (once)
* before this function is called.
*
+ * Upon successful return from this function the application has to perform at
+ * least one of the following operations within \c transaction.timeout.ms to
+ * avoid timing out the transaction on the broker:
+ * * rd_kafka_produce() (et.al)
+ * * rd_kafka_send_offsets_to_transaction()
+ * * rd_kafka_commit_transaction()
+ * * rd_kafka_abort_transaction()
+ *
* Any messages produced, offsets sent (rd_kafka_send_offsets_to_transaction()),
confluent-kafka-go applied the same move in confluentinc/confluent-kafka-go@89b6c2c
("Move doc from InitTransactions to BeginTransaction",
confluentinc/confluent-kafka-go#1095, 2024-01-10):
--- a/kafka/producer.go
+++ b/kafka/producer.go
@@ -807,14 +807,6 @@
// will acquire the internal producer id and epoch, used in all future
// transactional messages issued by this producer instance.
//
-// Upon successful return from this function the application has to perform at
-// least one of the following operations within `transaction.timeout.ms` to
-// avoid timing out the transaction on the broker:
-// - `Produce()` (et.al)
-// - `SendOffsetsToTransaction()`
-// - `CommitTransaction()`
-// - `AbortTransaction()`
-//
// Parameters:
@@ -846,6 +838,14 @@
// `InitTransactions()` must have been called successfully (once)
// before this function is called.
//
+// Upon successful return from this function the application has to perform at
+// least one of the following operations within `transaction.timeout.ms` to
+// avoid timing out the transaction on the broker:
+// - `Produce()` (et.al)
+// - `SendOffsetsToTransaction()`
+// - `CommitTransaction()`
+// - `AbortTransaction()`
+//
// Any messages produced, offsets sent (`SendOffsetsToTransaction()`),
So C and Go agree; Python and .NET are the two clients still carrying it in the
old place.
I can submit a PR for this.
- 主要言語
- Python
- スター
- 528
- フォーク
- 977
- 平均マージ
- 2日 19時間
- マージ済み PR(30日)
- 19
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートあり
- コントリビューションガイドなし
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
confluentinc/confluent-kafka-python のほかの issue
-
KIP-848 migration guide says never use assign()/unassign() with group.protocol='consumer'; it's only rejected after subscribe対応中かも @hidingmyname が 22 日前に担当しました。 オープン
難易度 1/5 1〜3時間 初心者へのやさしさ 88/100
confluentinc/confluent-kafka-python#2405 ·
メンテナーはふだん 3 日以内に返信
-
Wrong type annotations for read only mapping types.対応中かも @RasmusNygren が 198 日前に担当しました。 オープンstatus:under-review
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
confluentinc/confluent-kafka-python#2244 ·
メンテナーはふだん 3 日以内に返信
-
Wrong signature for `KafkaError`'s 1st parameter in `.pyi`対応中かも @wgu9 が 159 日前に担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
confluentinc/confluent-kafka-python#2243 ·
メンテナーはふだん 3 日以内に返信
-
Use joserfc instead of authlib.jose対応中かも @yvaishalirao が 135 日前に担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 62/100
confluentinc/confluent-kafka-python#2242 · コメント 2 件 ·
メンテナーはふだん 3 日以内に返信
-
Producer.poll(timeout) blocks for full timeout after delivery callback fires (regression in 2.13.0)再び着手できるかも このイシューのプルリクエストはマージされずにクローズされました。 オープンcomponent:producer
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
confluentinc/confluent-kafka-python#2215 · コメント 1 件 · リアクション 3 件 ·
メンテナーはふだん 3 日以内に返信
confluentinc/confluent-kafka-python の issue をすべて見る
似ている issue
-
needs-triage
難易度 1/5 1時間未満 初心者へのやさしさ 88/100
flashinfer-ai/flashinfer#9803 ·
メンテナーはふだん 1 日以内に返信
-
bug frontend good first issue help wanted
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
oscarbol09/thesisforge#61 ·
-
api bug performance
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
oscarbol09/audiobard#159 ·
-
area:web p2 product:business triage:active type:bug
難易度 1/5 1時間未満 初心者へのやさしさ 89/100
cuadrao-ai/argus#2328 ·
メンテナーはふだん 1 日以内に返信
-
bug frontend good first issue help wanted
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
nikhilb2/simple_invoicing#907 ·