Running into exception while using cloudevent values in kafka streams
Dieses Issue hat noch niemand übernommen.
Bewertung
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Anfängerfreundlichkeit
- 25/100
- Issue-Typ
- Bug
- Klarheit
- Muss geklärt werden
- Aktivitätsstatus
- Veraltet
- Tech-Stack
- java, kafka
- Bereich
- stream-processing
Rechercherichtung
Beginne damit, das Java-Beispiel mit CloudEvents Kafka 2.5.0 und Kafka Streams 2.8.0 zu reproduzieren, und untersuche anschließend CloudEventSerializer sowie den im Stacktrace gezeigten Serialisierungspfad des Kafka Streams state-store. Ermittle, ob die gemeldete Serde-Konfiguration für die KTable und den Join unterstützt wird; abgeschlossen ist die Aufgabe, wenn eine bestätigte Lösung oder eine dokumentierte, getestete Änderung vorliegt.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
I want to take join between a kafka stream and a ktable. The poc works fine with stream data. However, when I use CloudEvent, I keep running into some or other issue related to serialization.
Here is my code sample -
Map<String, Object> ceSerializerConfigs = new HashMap<>();
ceSerializerConfigs.put(ENCODING_CONFIG, Encoding.STRUCTURED);
ceSerializerConfigs.put(EVENT_FORMAT_CONFIG, JsonFormat.CONTENT_TYPE);
CloudEventSerializer serializer = new CloudEventSerializer();
serializer.configure(ceSerializerConfigs, false);
CloudEventDeserializer deserializer = new CloudEventDeserializer();
deserializer.configure(ceSerializerConfigs, false);
Serde<CloudEvent> cloudEventSerde = Serdes.serdeFrom(serializer, deserializer);
KStream<String, CloudEvent> kStream = builder.stream("stream-topic", Consumed.with(Serdes.String(), cloudEventSerde));
KTable<String, CloudEvent> kTable = builder.table("ktable-topic", Consumed.with(Serdes.String(), cloudEventSerde));
KStream<String, CloudEvent> joined = kStream
.join(kTable, (left, right) -> CloudEventBuilder.v1().withId(left.getId().concat(right.getId())).build());
joined.to(output, Produced.with(Serdes.String(), eventsSerde));
KafkaStreams kafkaStreams = new KafkaStreams(builder.build(), streamProps);
kafkaStreams.start();
I also tried using WrapperSerde - Issue with configuring Serdes for Kafka Streams
However I keep running into exception -
18:12:08.691 [basic-streams-updated-0630c691-0080-4e02-8c85-7bff650f34e9-StreamThread-1] ERROR org.apache.kafka.streams.KafkaStreams - stream-client [basic-streams-updated-0630c691-0080-4e02-8c85-7bff650f34e9] Encountered the following exception during processing and the registered exception handler opted to SHUTDOWN_CLIENT. The streams client is going to shut down now. org.apache.kafka.streams.errors.StreamsException: Exception caught in process. taskId=0_0, processor=KSTREAM-SOURCE-0000000002, topic=cloudevent-ktable, partition=0, offset=80, stacktrace=java.lang.UnsupportedOperationException: CloudEventSerializer supports only the signature serialize(String, Headers, CloudEvent)
Caused by: java.lang.UnsupportedOperationException: CloudEventSerializer supports only the signature serialize(String, Headers, CloudEvent) at io.cloudevents.kafka.CloudEventSerializer.serialize(CloudEventSerializer.java:84) ~[cloudevents-kafka-2.5.0.jar:?] at io.cloudevents.kafka.CloudEventSerializer.serialize(CloudEventSerializer.java:38) ~[cloudevents-kafka-2.5.0.jar:?] at org.apache.kafka.streams.state.internals.ValueAndTimestampSerializer.serialize(ValueAndTimestampSerializer.java:82) ~[kafka-streams-2.8.0.jar:?] at
org.apache.kafka.streams.state.internals.ValueAndTimestampSerializer.serialize(ValueAndTimestampSerializer.java:73) ~[kafka-streams-2.8.0.jar:?] at
org.apache.kafka.streams.state.internals.ValueAndTimestampSerializer.serialize(ValueAndTimestampSerializer.java:30) ~[kafka-streams-2.8.0.jar:?] at org.apache.kafka.streams.state.StateSerdes.rawValue(StateSerdes.java:192) ~[kafka-streams-2.8.0.jar:?] at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.lambda$put$4(MeteredKeyValueStore.java:200) ~[kafka-streams-2.8.0.jar:?] at org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:884) ~[kafka-streams-2.8.0.jar:?] at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.put(MeteredKeyValueStore.java:200) ~[kafka-streams-2.8.0.jar:?] at org.apache.kafka.streams.processor.internals.AbstractReadWriteDecorator$KeyValueStoreReadWriteDecorator.put(AbstractReadWriteDecorator.java:120) ~[kafka-streams-2.8.0.jar:?] at org.apache.kafka.streams.kstream.internals.KTableSource$KTableSourceProcessor.process(KTableSource.java:122) ~[kafka-streams-2.8.0.jar:?]
18:12:08.691 [basic-streams-updated-0630c691-0080-4e02-8c85-7bff650f34e9-StreamThread-1] ERROR org.apache.kafka.streams.KafkaStreams - stream-client [basic-streams-updated-0630c691-0080-4e02-8c85-7bff650f34e9] Encountered the following exception during processing and the registered exception handler opted to SHUTDOWN_CLIENT. The streams client is going to shut down now. org.apache.kafka.streams.errors.StreamsException: Exception caught in process. taskId=0_0, processor=KSTREAM-SOURCE-0000000002, topic=cloudevent-ktable, partition=0, offset=80, stacktrace=java.lang.UnsupportedOperationException: CloudEventSerializer supports only the signature serialize(String, Headers, CloudEvent)
Caused by: java.lang.UnsupportedOperationException: CloudEventSerializer supports only the signature serialize(String, Headers, CloudEvent) at io.cloudevents.kafka.CloudEventSerializer.serialize(CloudEventSerializer.java:84) ~[cloudevents-kafka-2.5.0.jar:?] at io.cloudevents.kafka.CloudEventSerializer.serialize(CloudEventSerializer.java:38) ~[cloudevents-kafka-2.5.0.jar:?] at org.apache.kafka.streams.state.internals.ValueAndTimestampSerializer.serialize(ValueAndTimestampSerializer.java:82) ~[kafka-streams-2.8.0.jar:?] at
org.apache.kafka.streams.state.internals.ValueAndTimestampSerializer.serialize(ValueAndTimestampSerializer.java:73) ~[kafka-streams-2.8.0.jar:?] at
org.apache.kafka.streams.state.internals.ValueAndTimestampSerializer.serialize(ValueAndTimestampSerializer.java:30) ~[kafka-streams-2.8.0.jar:?] at org.apache.kafka.streams.state.StateSerdes.rawValue(StateSerdes.java:192) ~[kafka-streams-2.8.0.jar:?] at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.lambda$put$4(MeteredKeyValueStore.java:200) ~[kafka-streams-2.8.0.jar:?] at org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:884) ~[kafka-streams-2.8.0.jar:?] at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.put(MeteredKeyValueStore.java:200) ~[kafka-streams-2.8.0.jar:?] at org.apache.kafka.streams.processor.internals.AbstractReadWriteDecorator$KeyValueStoreReadWriteDecorator.put(AbstractReadWriteDecorator.java:120) ~[kafka-streams-2.8.0.jar:?] at org.apache.kafka.streams.kstream.internals.KTableSource$KTableSourceProcessor.process(KTableSource.java:122) ~[kafka-streams-2.8.0.jar:?]
Am I missing anything, any help from CloudEvent team is appreciated.
- Vorherrschende Sprache
- Java
- Sterne
- 447
- Forks
- 173
- Ø Merge
- 16 Std. 42 Min.
- Gemergte PRs (30 T.)
- 2
Entwicklungsumgebung
- Kein Dockerfile und keine Docker-Compose-Datei
- Keine Pull-Request-Vorlage
- Beitragsleitfaden lesen
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus cloudevents/sdk-java
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
cloudevents/sdk-java#731 · 1 Reaktion ·
-
Avro compact deserializer bug - ID set to typeEvtl. wieder frei @Taimoo-r hat das vor 78 Tagen übernommen, und es ist kein Pull Request offen. Offenbug
Schwierigkeit 1/5 1-3 Stunden Anfängerfreundlichkeit 88/100
cloudevents/sdk-java#697 · 3 Kommentare ·
-
Spring CloudEventMessageConverter produce a null pointer exception if initial message header map is nullEvtl. vergeben @prashantdotnaagar hat das vor 95 Tagen übernommen. Offen
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
cloudevents/sdk-java#689 · 2 Kommentare ·
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 65/100
cloudevents/sdk-java#660 · 2 Kommentare ·
-
cloudevents-bom pulls in jakarta.jakartaee-bom from the parentEvtl. vergeben @AdzerKI hat das vor 14 Tagen übernommen. Offen
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 25/100
cloudevents/sdk-java#749 ·
Alle Issues in cloudevents/sdk-java
Ähnliche Issues
-
enhancement feature
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 70/100
rsp777/service-pulse-app#28 ·
-
human-required sync-failed
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 68/100
Azure/osdu-spi-file#63 ·
Maintainer antworten meist innerhalb von 1 Tag
-
area/ysql kind/enhancement priority/medium status/awaiting-triage
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 73/100
yugabyte/yugabyte-db#35578 ·
Maintainer antworten meist innerhalb von 1 Tag
-
frontend quick win
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 65/100
CodeForPhilly/benefit-decision-toolkit#571 ·
Maintainer antworten meist innerhalb von 1 Tag
-
approved
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 67/100
checkstyle/checkstyle#22129 ·
Maintainer antworten meist innerhalb von 1 Tag