Add method to get the schema (version) used for serialization of a message

Open

#362 opened on Jan 28, 2024 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 5/5
Estimated time: Over a week
Newbie friendliness: 25/100

Issue type: Feature
Clarity: Needs clarification
Activity status: Stale
Tech stack: javascript, nodejs
Domain: distributed-systems

Research direction

Start with the Pulsar schema overview and the linked node-client issue to verify how schema versions are attached to messages and whether the Node client exposes them. Define the API needed to retrieve the serialization schema or version, then confirm that it supports reliable Avro deserialization across schema evolution.

Written by the indexing model from the issue text.

Description

There are client libraries for other languages (e.g. java) that support automatic (de)serialization of message data based on a schema (https://pulsar.apache.org/docs/3.1.x/schema-overview/).
The node client does not have this feature yet (https://github.com/apache/pulsar-client-node/issues/242).

In theory it should be possible to deserialize the message data manually, BUT...
Some serialization formats like Avro require to know the exact schema that was used for serialization (https://github.com/mtth/avsc/issues/447). In order to deal with schema evolution the client needs to know it's own compatible schema AND the schema used for serialization.
As far I understand, the automatic (de)serialization feature of Pulsar solves this problem by keeping a schema registry and tagging the messages with the used schema version.
If I understand right, the node client does not provide a method to get the schema used for serialization and not even a method to get the schema version of a message.
Assuming the need for schema evolution, this makes it impossible to deserialize reliably messages written by a java client library using an Avro schema,

I'm new to Pulsar and Avro, so please forgive (and correct) me if my understanding is wrong.

If my understanding is right, I wonder how difficult it would be to add a method to lookup the serialization schema on an message.

Dominant language: C++
Stars: 164
Forks: 99
Avg merge: 5d 18h
Merged PRs (30d): 2

Contributor guide

No contributing guide indexed for this repository

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

Use 4:4:4 for multichannel AVIF and HEIF gain maps Open

Difficulty 2/5 1-3 hours Newbie friendliness 70/100

google/libultrahdr#485 · Opened Sep 24, 2026
Missing tile warnings in TileMap dock display with wrong texture filter Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

godotengine/godot#123776 · Opened Sep 23, 2026
if(cond,a,b) treats any nonzero cond as true in every bngsim backend (ExprTk interpreter, C codegen, SSA), which contradicts docs/reference/expressions.md ('true when > 0.5') and BNG's run_network. A non-boolean condition in (-inf,0) or... Open

bug

Difficulty 1/5 Under an hour Newbie friendliness 60/100

lanl/bngsim#774 · Opened Sep 23, 2026
Delete ament_export_include_directories() and ament_export_libraries() calls Open

good first issue

Difficulty 1/5 Under an hour Newbie friendliness 90/100

ros2/rosbag2#2509 · Opened Sep 23, 2026
Delete ament_export_include_directories() and ament_export_libraries() calls Open

good first issue

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

ros2/common_interfaces#344 · Opened Sep 23, 2026

More C++ issues