apache/pulsar

Kafka Sink Connector: support non-String keys

Open

#9,848 opened on Mar 9, 2021

View on GitHub
 (4 comments) (0 reactions) (0 assignees)Java (3,485 forks)batch import
good first issuehelp wantedtype/enhancement

Repository metrics

Stars
 (13,697 stars)
PR merge metrics
 (Avg merge 3d 10h) (146 merged PRs in 30d)

Description

Currently the Kafka Sink Connector is able to support only keys with a String value.

We should support the generic byte[] key, as it is in Kafka. Also we should support other data types and possibly may the generic key/value pair of Kafka to a KeyValue Pulsar schema

Contributor guide

Research direction
Investigate the current Kafka Sink Connector implementation for key handling, understand the schema registry and serialization mechanisms in both Kafka and Pulsar, and propose a solution to support non String keys (e.g., byte[] or other types) using appropriate schema conversions.
Tech stack
java
Domain
backend
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
JavaKafkaPulsar
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40