In `avro-enable-watermark=true` using AWS Glue Schema Registry, "DELETE" event and normal event has colliding magic byte

Open Beginner friendly

#5,552 opened on Jul 1, 2026 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 2/5
Estimated time: 1-3 hours
Newbie friendliness: 72/100

Issue type: Bug
Clarity: Clearly specified
Activity status: Quiet
Tech stack: aws, go
Domain: backend, data-engineering

Research direction

Start with the referenced byte definitions in pkg/sink/codec/avro/helper.go and pkg/sink/codec/avro/glue_schema_registry.go, then trace the identification logic in pkg/sink/codec/avro/decoder.go. Confirm which internal-use bytes collide with AWS Glue's header version byte, update the internal values without overlapping Glue messages, and verify that DELETE and normal events are distinguished when avro-enable-watermark and AWS Glue Schema Registry are both enabled.

Written by the indexing model from the issue text.

Description

deleteByte is 3

https://github.com/pingcap/ticdc/blob/41b855b10b129a6b4af681a6f1b02362f6e4f104/pkg/sink/codec/avro/helper.go#L48-L55

The headerVersionByte of a Glue message is also 3

https://github.com/pingcap/ticdc/blob/41b855b10b129a6b4af681a6f1b02362f6e4f104/pkg/sink/codec/avro/glue_schema_registry.go#L325-L332

This will lead to mis-identification when both avro-enable-watermark and AWS Glue Schema Registry are used, e.g.

https://github.com/pingcap/ticdc/blob/41b855b10b129a6b4af681a6f1b02362f6e4f104/pkg/sink/codec/avro/decoder.go#L194-L199

I suggest changing the 3 internal-use bytes to some value far away from these magic bytes, and probably don't occupy three slots for internal use. Maybe change to e.g. cd c1, cd c2, cd c3.

(Internal reference: GTOC-8564/NAID-12126)

Dominant language: Go
Stars: 56
Forks: 63
Avg merge: 2d 17h
Merged PRs (30d): 43

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

GrafeasNote is a duplicate resource of ContainerAnalysisNote Open

Difficulty 1/5 Under an hour Newbie friendliness 84/100

GoogleCloudPlatform/k8s-config-connector#13349 · Opened Sep 21, 2026
[receiver/receiverhelper] skip WithLinks when the parent span context is invalid Open

enhancement needs triage

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

open-telemetry/opentelemetry-collector#15998 · Opened Sep 21, 2026
🧹 Use PodSet wrappers in DRA tests Open

kind/cleanup

Difficulty 2/5 1-3 hours Newbie friendliness 88/100

kubernetes-sigs/kueue#15947 · Opened Sep 21, 2026
Custom provider credential is written as `PROVIDER_API_KEY` (apiserver) but the AgentRun controller injects only allowlisted keys, so every custom provider fails with HTTP 401 Open

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

sympozium-ai/sympozium#627 · Opened Sep 21, 2026
A recording that opens with an all-zero last_throw emits a zeroed discThrown at frame 0 Open

Difficulty 2/5 1-3 hours Newbie friendliness 86/100

EchoTools/tape#60 · Opened Sep 21, 2026

More Go issues