KIP-848 migration guide says never use assign()/unassign() with group.protocol='consumer'; it's only rejected after subscribe

Open Beginner friendly
#2,405 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 3 days

@hidingmyname is already working on this.

Since Oct 29, 2026.

  • #2407 by @hidingmyname — open

Assessment

Difficulty
1/5
Estimated time
1-3 hours
Newbie friendliness
88/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active
Tech stack
kafka, python
Domain
documentation

Research direction

Start with the “Rebalance Callback Changes” section in docs/kip-848-migration-guide.md and its matching section in docs/kip-848-migration-guide.rst. Compare the wording with the behavior described in the issue and the .NET guide’s “Manual partition assignment” section. Done means both Python guides clarify that manual assign() is supported before subscribing and describe the restriction after subscribing.

Written by the indexing model from the issue text.

Description

Description

docs/kip-848-migration-guide.md, section "Rebalance Callback Changes":

- **Do not** use `consumer.assign()` or `consumer.unassign()` when using `group.protocol='consumer'` (KIP-848).

docs/kip-848-migration-guide.rst has the same text:

-  **Do not** use ``consumer.assign()`` or ``consumer.unassign()`` when
   using ``group.protocol='consumer'`` (KIP-848).

This is broader than what the client actually enforces. In librdkafka, the
check that rejects a non-incremental assign ("Changes to the current
assignment must be made using incremental_assign() or incremental_unassign()
when rebalance protocol type is COOPERATIVE") only applies once the rebalance
protocol is COOPERATIVE. With the consumer protocol, that only happens
after the consumer has subscribed. Before that, rd_kafka_cgrp_rebalance_protocol()
returns NONE:

if (rkcg->rkcg_group_protocol == RD_KAFKA_GROUP_PROTOCOL_CONSUMER) {
        if (!(rkcg->rkcg_consumer_flags &
              RD_KAFKA_CGRP_CONSUMER_F_SUBSCRIBED_ONCE))
                return RD_KAFKA_REBALANCE_PROTOCOL_NONE;

        return RD_KAFKA_REBALANCE_PROTOCOL_COOPERATIVE;
}

So consumer.assign() for manual partition assignment, without calling
subscribe(), still works with group.protocol='consumer'.

Why it matters

Applications that use manual assignment, often with a group.id set only for
committing offsets, read this guide to decide whether they can switch
group.protocol. As written, the guide tells them they can't use assign()
at all, which isn't true. The restriction that actually matters, using only
incremental_assign()/incremental_unassign() once subscribed, gets lost
in the over-broad wording.

Suggested fix
-- **Do not** use `consumer.assign()` or `consumer.unassign()` when using `group.protocol='consumer'` (KIP-848).
+- **Do not** use `consumer.assign()` or `consumer.unassign()` after subscribing with `group.protocol='consumer'` (KIP-848).
+  `consumer.assign()` can still be used for manual partition assignment without subscribing.

and the same change in docs/kip-848-migration-guide.rst:

--  **Do not** use ``consumer.assign()`` or ``consumer.unassign()`` when
-   using ``group.protocol='consumer'`` (KIP-848).
+-  **Do not** use ``consumer.assign()`` or ``consumer.unassign()`` after
+   subscribing with ``group.protocol='consumer'`` (KIP-848).
+   ``consumer.assign()`` can still be used for manual partition assignment
+   without subscribing.
Upstream reference

confluent-kafka-go fixed the same sentence in its copy of this guide in
confluentinc/confluent-kafka-go@cc3d3f7 ("Few minor nits", 2025-10-10):

--- a/kafka/kip-848-migration-guide.md
+++ b/kafka/kip-848-migration-guide.md
@@ -51,15 +51,15 @@
 - In the **rebalance callbacks**, you **must only use** (optional - if not used, client will handle it internally):
   - `consumer.IncrementalAssign(e.Partitions)` to assign new partitions
   - `consumer.IncrementalUnassign(e.Partitions)` to revoke partitions
-- **Do not** use `consumer.Assign()` or `consumer.Unassign()` when using `group.protocol='consumer'` (KIP-848).
+- **Do not** use `consumer.Assign()` or `consumer.Unassign()` after subscribing with `group.protocol='consumer'` (KIP-848).

The .NET guide (doc/kip-848-migration-guide.md) already describes it
correctly in its "Manual partition assignment" section: "You can still use
Assign() before being subscribed but after subscribing you can only use
IncrementalAssign() and IncrementalUnassign()." All three clients use
the same librdkafka check, so the Python guide can use the same wording.

I can submit a PR for this.

Dominant language
Python
Stars
528
Forks
977
Avg merge
2d 19h
Merged PRs (30d)
19

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from confluentinc/confluent-kafka-python

All issues in confluentinc/confluent-kafka-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.