[Bug] Default key routing (JavaStringHash) picks a different partition than the Java client for about half of keys

Open Beginner friendly
#1,575 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

@0z5a is already working on this.

Since Nov 22, 2026.

  • #1576 by @0z5a — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
82/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
cpp, go, java, python

Research direction

The bug is in internal.JavaStringHash, which returns the raw 32-bit hash without clearing the sign bit. NewDefaultRouter and the single-partition router then take it modulo the partition count. Compare with Murmur3_32Hash in the same file, which masks with 0x7fffffff, and check how the default router consumes the value. Done when keys such as customer-42 and device-7f3a map to the partitions in the issue's Java column for 3, 5, 6 and 10 partitions, and existing hash tests still pass.

Written by the indexing model from the issue text.

Description

Expected behavior

With the default HashingScheme (JavaStringHash), a message with a given key should go to the same partition as the same key sent by the Java client (and the C++/Python clients). The option is documented as "Java String.hashCode() equivalent".

Actual behavior

The Java client routes a keyed message to signSafeMod(key.hashCode() & Integer.MAX_VALUE, numPartitions); the C++ client also clears the sign bit (hash &= std::numeric_limits<int32_t>::max()). The Go internal.JavaStringHash returns the raw 32-bit value, and NewDefaultRouter / the single-partition router then compute hash % numPartitions on it.

When hashCode() has the top bit set, the Go result differs unless the partition count is a power of two. Murmur3_32Hash in the same file already masks with 0x7fffffff "to maintain compatibility with values used in Java client"; JavaStringHash does not.

Steps to reproduce

Partition chosen for a keyed message, for 3 / 5 / 6 / 10 partitions. Java: RoundRobinPartitionMessageRouterImpl(HashingScheme.JavaStringHash, …).choosePartition from pulsar-client 5.0.1. Go: pulsar.NewDefaultRouter(internal.JavaStringHash, …) at master 05daf3d.

key Java 5.0.1 Go master
hello 1 2 4 2 1 2 4 2
order-1001 2 3 5 3 2 3 5 3
customer-42 1 3 1 3 0 1 3 1
device-7f3a 0 1 0 6 2 4 2 4

"customer-42".hashCode() is 0x965fbacd. Java's makeHash returns 0x165fbacd, but Go's JavaStringHash returns 0x965fbacd.

Over 100,000 random UUID-string keys, 49,749 land on a different partition than in the Java client for every partition count tested that is not a power of two (3, 5, 6, 10, 12). With 16 partitions, none do.

Where this shows up

When a topic is produced to from both Go and Java (or C++/Python) services, or a producer is ported from Java to Go, the same key is written to two partitions. Per-key ordering is then lost, and consumers that rely on key → partition affinity (Failover/Exclusive per partition, stateful processing keyed by partition) see a key's messages split.

Note: non-ASCII keys differ for a second reason. Go hashes UTF-8 bytes, while Java hashes UTF-16 code units (for example café). That is a separate question, and this report covers only the sign-bit handling, which affects plain ASCII keys.

System configuration

Pulsar version: client master (05daf3d); Java reference pulsar-client 5.0.1

Dominant language
Go
Stars
746
Forks
394
Avg merge
6d 13h
Merged PRs (30d)
7

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/pulsar-client-go

All issues in apache/pulsar-client-go

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.