KIP-848 migration guide says never use assign()/unassign() with group.protocol='consumer'; it's only rejected after subscribe
Maintainers usually reply within 3 days
Assessment
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 88/100
- Issue type
- Documentation
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- kafka, python
- Domain
- documentation
Research direction
Start with the “Rebalance Callback Changes” section in docs/kip-848-migration-guide.md and its matching section in docs/kip-848-migration-guide.rst. Compare the wording with the behavior described in the issue and the .NET guide’s “Manual partition assignment” section. Done means both Python guides clarify that manual assign() is supported before subscribing and describe the restriction after subscribing.
Written by the indexing model from the issue text.
Description
Description
docs/kip-848-migration-guide.md, section "Rebalance Callback Changes":
- **Do not** use `consumer.assign()` or `consumer.unassign()` when using `group.protocol='consumer'` (KIP-848).
docs/kip-848-migration-guide.rst has the same text:
- **Do not** use ``consumer.assign()`` or ``consumer.unassign()`` when
using ``group.protocol='consumer'`` (KIP-848).
This is broader than what the client actually enforces. In librdkafka, the
check that rejects a non-incremental assign ("Changes to the current
assignment must be made using incremental_assign() or incremental_unassign()
when rebalance protocol type is COOPERATIVE") only applies once the rebalance
protocol is COOPERATIVE. With the consumer protocol, that only happens
after the consumer has subscribed. Before that, rd_kafka_cgrp_rebalance_protocol()
returns NONE:
if (rkcg->rkcg_group_protocol == RD_KAFKA_GROUP_PROTOCOL_CONSUMER) {
if (!(rkcg->rkcg_consumer_flags &
RD_KAFKA_CGRP_CONSUMER_F_SUBSCRIBED_ONCE))
return RD_KAFKA_REBALANCE_PROTOCOL_NONE;
return RD_KAFKA_REBALANCE_PROTOCOL_COOPERATIVE;
}
So consumer.assign() for manual partition assignment, without calling
subscribe(), still works with group.protocol='consumer'.
Why it matters
Applications that use manual assignment, often with a group.id set only for
committing offsets, read this guide to decide whether they can switch
group.protocol. As written, the guide tells them they can't use assign()
at all, which isn't true. The restriction that actually matters, using only
incremental_assign()/incremental_unassign() once subscribed, gets lost
in the over-broad wording.
Suggested fix
-- **Do not** use `consumer.assign()` or `consumer.unassign()` when using `group.protocol='consumer'` (KIP-848).
+- **Do not** use `consumer.assign()` or `consumer.unassign()` after subscribing with `group.protocol='consumer'` (KIP-848).
+ `consumer.assign()` can still be used for manual partition assignment without subscribing.
and the same change in docs/kip-848-migration-guide.rst:
-- **Do not** use ``consumer.assign()`` or ``consumer.unassign()`` when
- using ``group.protocol='consumer'`` (KIP-848).
+- **Do not** use ``consumer.assign()`` or ``consumer.unassign()`` after
+ subscribing with ``group.protocol='consumer'`` (KIP-848).
+ ``consumer.assign()`` can still be used for manual partition assignment
+ without subscribing.
Upstream reference
confluent-kafka-go fixed the same sentence in its copy of this guide in
confluentinc/confluent-kafka-go@cc3d3f7 ("Few minor nits", 2025-10-10):
--- a/kafka/kip-848-migration-guide.md
+++ b/kafka/kip-848-migration-guide.md
@@ -51,15 +51,15 @@
- In the **rebalance callbacks**, you **must only use** (optional - if not used, client will handle it internally):
- `consumer.IncrementalAssign(e.Partitions)` to assign new partitions
- `consumer.IncrementalUnassign(e.Partitions)` to revoke partitions
-- **Do not** use `consumer.Assign()` or `consumer.Unassign()` when using `group.protocol='consumer'` (KIP-848).
+- **Do not** use `consumer.Assign()` or `consumer.Unassign()` after subscribing with `group.protocol='consumer'` (KIP-848).
The .NET guide (doc/kip-848-migration-guide.md) already describes it
correctly in its "Manual partition assignment" section: "You can still use
Assign() before being subscribed but after subscribing you can only use
IncrementalAssign() and IncrementalUnassign()." All three clients use
the same librdkafka check, so the Python guide can use the same wording.
I can submit a PR for this.
- Dominant language
- Python
- Stars
- 528
- Forks
- 977
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 19
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from confluentinc/confluent-kafka-python
-
init_transactions docstring says the transaction.timeout.ms clock starts there; it starts at begin_transactionPossibly taken @hidingmyname claimed this 22 days ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 91/100
confluentinc/confluent-kafka-python#2404 ·
Maintainers usually reply within 3 days
-
Wrong type annotations for read only mapping types.Possibly taken @RasmusNygren claimed this 198 days ago. Openstatus:under-review
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
confluentinc/confluent-kafka-python#2244 ·
Maintainers usually reply within 3 days
-
Wrong signature for `KafkaError`'s 1st parameter in `.pyi`Possibly taken @wgu9 claimed this 159 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
confluentinc/confluent-kafka-python#2243 ·
Maintainers usually reply within 3 days
-
Use joserfc instead of authlib.josePossibly taken @yvaishalirao claimed this 135 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
confluentinc/confluent-kafka-python#2242 · 2 comments ·
Maintainers usually reply within 3 days
-
Producer.poll(timeout) blocks for full timeout after delivery callback fires (regression in 2.13.0)May be free again A pull request for this issue was closed without being merged. Opencomponent:producer
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
confluentinc/confluent-kafka-python#2215 · 1 comment · 3 reactions ·
Maintainers usually reply within 3 days
All issues in confluentinc/confluent-kafka-python
Similar issues
-
needs-triage
Difficulty 1/5 Under an hour Newbie friendliness 88/100
flashinfer-ai/flashinfer#9803 ·
Maintainers usually reply within 1 day
-
bug frontend good first issue help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
oscarbol09/thesisforge#61 ·
-
api bug performance
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
oscarbol09/audiobard#159 ·
-
area:web p2 product:business triage:active type:bug
Difficulty 1/5 Under an hour Newbie friendliness 89/100
cuadrao-ai/argus#2328 ·
Maintainers usually reply within 1 day
-
bug frontend good first issue help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
nikhilb2/simple_invoicing#907 ·