[Bug] A cleared group forbidden state is persisted as the literal "null" and then fails the config load on the next broker start
维护者通常 1 天内回复
评估
调研方向
Start in broker/src/main/java/org/apache/rocketmq/broker/config/v1/RocksDBSubscriptionGroupManager.java, where updateForbidden, setForbidden and clearForbidden write JSON.toJSONString of the group's table entry, and in decodeForbidden, which dereferences the parsed body. Run RocksDBSubscriptionGroupManagerTest to see the "null" write and the decode failure. Done means the cleared key is removed and a null body is skipped on load. Linked PR #11430 is already open, so check it before starting.
由索引模型根据 Issue 内容生成。
描述
Before Creating the Bug Report
- I found a bug, not just asking a question, which should be created in GitHub Discussions.
- I have searched the GitHub Issues and GitHub Discussions of this repository and believe that this is not a duplicate.
- I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ.
Runtime platform environment
OS: any (broker with the RocksDB config manager: enableRocksDBStore=true, i.e. RocksDBSubscriptionGroupManager)
RocketMQ version
branch: develop
Git commit id: a7edeecfb
JDK Version
Compiler: Temurin 1.8.0_504 (target 1.8, same as CI)
Describe the Bug
The RocksDB variant of the subscription group manager persists the group's forbidden bits with JSON.toJSONString(this.getForbiddenTable().get(group)), and after the last bit of a group is cleared the table no longer holds an entry for that group — so the value written to the forbidden column family is the string "null":
@Override
public void updateForbidden(String group, String topic, int forbiddenIndex, boolean setOrClear) {
super.updateForbidden(group, topic, forbiddenIndex, setOrClear);
this.rocksDBConfigManager.put(FORBIDDEN_COLUMN_FAMILY_NAME, group,
JSON.toJSONString(this.getForbiddenTable().get(group))); // -> "null" once the group has no forbidden bits left
}
The same shape appears in setForbidden and clearForbidden (broker/src/main/java/org/apache/rocketmq/broker/config/v1/RocksDBSubscriptionGroupManager.java:287-306; the updateForbidden variant is at :276-284).
On the next start the stored value is fed back into the loader, which expects a JSON object of topic → bits:
protected void decodeForbidden(byte[] key, byte[] body) {
JSONObject jsonObject = JSON.parseObject(new String(body, RocksDBConfigManager.CHARSET));
Set<Map.Entry<String, Object>> entries = jsonObject.entrySet(); // NPE for a "null" body
...
}
(:263-273.) JSON.parseObject("null") returns null, so the NullPointerException leaves loadForbidden through its catch and the method returns false (:101-109), loadSubscriptionGroupAndForbidden() returns false (:95-99), then RocksDBSubscriptionGroupManager.load() returns false (:81-83), BrokerController.initializeMetadata() returns false (BrokerController.java:863) and BrokerStartup answers that with System.exit(-3) (BrokerStartup.java:287-291) — the broker does not start until that key is repaired by hand.
The trigger needs nothing more than un-forbidding the last forbidden topic of a group; with a single forbidden topic that is the very first clear, so no multi-topic scenario is required. (The related cross-topic loss when a group has several forbidden topics is reported separately in #11093 — this report is about the value that gets persisted and what it breaks on the next load; the two fixes compose.)
Steps to Reproduce
- Start a broker with
enableRocksDBStore=trueand forbid a topic for a group, e.g. viaUPDATE_AND_GET_GROUP_FORBIDDEN(AdminBrokerProcessor.updateAndGetGroupForbidden,:466-489) withreadable=falseforgroup=g,topic=t. - Un-forbid the same topic (
readable=true). Theforbiddencolumn family ofgroupnow holds the literalnull. - Restart the broker: the config load fails and the process exits (
initialize()false →System.exit(-3)), withloadForbidden exception … java.lang.NullPointerExceptionatdecodeForbidden.
At the unit level, RocksDBSubscriptionGroupManagerTest already injects a mocked RocksDBConfigManager (no real RocksDB needed): forbidding then clearing one topic shows the "null" value being written today, and calling decodeForbidden(key, "null".getBytes()) throws.
What Did You Expect to See?
Clearing a group's forbidden state persists "no forbidden state": the key is removed (or at least stores a valid empty JSON object), and the loader tolerates an already written null body so that brokers which ran with the broken value can still start after upgrading.
What Did You See Instead?
A "null" value is written, and the next broker start aborts because the loader dereferences the parsed null; the failure mode is a broker that cannot start from its own persisted configuration.
Additional Context
Minimal fix in RocksDBSubscriptionGroupManager: route the three overrides through one helper that deletes the column-family key when the group has no forbidden bits left instead of writing JSON.toJSONString(null), and ignore an empty (null) body in decodeForbidden so existing stores recover. The sibling JSON mode persists the whole table, which is why it never hits this.
I will open a PR with those changes plus regression tests in RocksDBSubscriptionGroupManagerTest (red before: a "null" value is written / decodeForbidden throws; green after: the key is deleted and a null body is skipped).
- 主要语言
- Java
- 星标
- 22.7k
- 派生
- 12k
- 平均合并
- 4 天 20 小时
- 30 天内合并 PR
- 11
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
apache/rocketmq 的其他 Issue
-
[Bug] RemotingHelper.connect swaps the socket send and receive buffer configuration可能已有人在做 @amalsaidov 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 1 天内回复
-
CompressionType.of fails under the Turkish locale可能已有人在做 @sharanggupta 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 83/100
维护者通常 1 天内回复
-
[rocketmq-client-python] TransactionMQProducer allocates an unused producer during initialization未关闭
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 1 天内回复
-
[Bug] notifyMasterOnline aborts the long-polling wake-up with a NullPointerException when an arrival drains a bucket first可能已有人在做 @iloveeyjafjalla 于 29 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 82/100
apache/rocketmq#11441 · 1 条评论 ·
维护者通常 1 天内回复
-
[Enhancement] FilterFactory duplicate-registration and unregister semantics lack tests可能已有人在做 @chennaji9 于 34 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 84/100
apache/rocketmq#11340 · 2 条评论 ·
维护者通常 1 天内回复
相似的 Issue
-
bug emitter:client:csharp
难度 2/5 1-3 小时 新手友好度 84/100
microsoft/typespec#12572 · 1 条评论 · 1 个 reaction ·
维护者通常 1 天内回复
-
GCC 15.2: -Wcast-qual warning - cast discards const qualifier from pointer target type (H5trace.c)未关闭
难度 2/5 1-3 小时 新手友好度 73/100
维护者通常 1 天内回复
-
needs-area-label
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 2 天内回复
-
area/ai area/backend component/zeebe kind/task
难度 2/5 1-3 小时 新手友好度 68/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
spAnser/reacharound#38 ·