Restructure the Kafka related operators
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Refactor
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- java, kafka
- Domain
- distributed-systems
Research direction
Start by comparing the Kafka-related operators currently in wayang-java and wayang-spark, then review the wayang-platforms structure. The proposal names a new wayang-kafka module containing JavaKafkaOperator and SparkKafkaOperator. Done would mean Kafka implementations are centralized there and the respective platform modules no longer contain Kafka-specific operators.
Written by the indexing model from the issue text.
Description
Proposal:
We create a wayang-kafka module inside of wayang-platforms.
This leads to a cleaner approach to add these operators.
Right now we edit both wayang-java and wayang-spark, where both of them contain versions of operators related to reading and writing to Kafka.
Conceptually, having all implementations for the specific APIs in wayang-kafka would be neat.
Then you could have wayang-kafka/src/main/java/JavaKafkaOperator wayang-kafka/src/main/java/SparkKafkaOperator etc. and the respective platforms themselves don't have to concern about Kafka.
It is planned to implement this as soon as the initial demos for Kafka integration are ready.
Target: Release 1.1 - Q1-2025.
- Dominant language
- Java
- Stars
- 274
- Forks
- 141
- Avg merge
- 5d 16h
- Merged PRs (30d)
- 4
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/wayang
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
[Docs] Fix broken markdown formatting, leaked reviewer prompt, and outdated URLs in tutorial.md Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
convert to config Opentodo
Difficulty 3/5 1-2 days Newbie friendliness 45/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100