apache/pulsar

PIP 37-Follow up: Chunk message with batching enabled

Open

#7,138 opened on Jun 2, 2020

View on GitHub
 (1 comment) (1 reaction) (1 assignee)Java (3,485 forks)batch import
help wantedtype/feature

Repository metrics

Stars
 (13,697 stars)
PR merge metrics
 (Avg merge 3d 10h) (146 merged PRs in 30d)

Description

Motivation

#4400 provides client to publish large size message into chunks. However, it requires producer-client to disable batching explicitly. Enhance pulasr-client lib to chunk message without disabling the batching.

Contributor guide

Research direction
Examine the existing chunking implementation in Pulsar client (e.g., in C++ and Java), understand how batching works internally, and design a solution to enable chunking without disabling batching by modifying the producer's message handling pipeline.
Tech stack
java
Domain
distributed systems
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
JavaPulsar clientMessaging batching concept
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
65