CLI-3553: Avro JSON Encoding Union Type Fails to Decode
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 38/100
Research direction
Reproduce the failure with the provided Avro schema and sample record, comparing the CLI with /usr/bin/kafka-avro-console-producer in the Confluent Schema Registry container. Trace the CLI's JSON-encoded Avro decoding path and verify that the timestamp union value follows the Avro JSON encoding specification without producing the reported error.
Written by the indexing model from the issue text.
Description
When attempting to write records to Confluent Cloud using the CLI, we sometimes get errors related JSON Encoded Avro records that deal with union types.
According to the JSON Encoding spec from Avro, when using a union type
- if its type is null, then it is encoded as a JSON null;
- otherwise it is encoded as a JSON object with one name/value pair whose name is the type’s name and whose value is the recursively encoded value. For Avro’s named types (record, fixed or enum) the user-specified name is used, for other types the type name is used.
We can do this locally using the included /usr/bin/kafka-avro-console-producer inside of the CP Schema Registry docker container, but not using the CLI. Instead, we get
Error: cannot decode textual record "<record-namespace>.<record-name>": cannot decode textual union: cannot decode textual map: cannot determine codec: "<type>" for key: "<field-name>"
Below is the full schema (with some sensitive bits removed) and the sample record we are using
{
"type" : "record",
"name" : "Updated",
"namespace" : "namespace",
"doc" : "Updated Event",
"fields" : [ {
"name" : "subjectivity_id",
"type" : {
"type" : "string",
"logicalType" : "string"
},
"doc" : "Identifier"
}, {
"name" : "topic",
"type" : "string",
"doc" : "Topic name"
}, {
"name" : "created_at",
"type" : [ "null", {
"type" : "long",
"logicalType" : "timestamp-millis"
} ],
"doc" : "Date indicating when the record was created",
"default" : null
}, {
"name" : "updated_at",
"type" : [ "null", {
"type" : "long",
"logicalType" : "timestamp-millis"
} ],
"doc" : "Date indicating when the record was updated",
"default" : null
}]
}
{ "subjectivity_id": "subj-1234","topic": "subjectivity_updated","created_at": null,"updated_at": { "long": 1696613296 }}
Let me know what other data we can provide!
- Dominant language
- Go
- Stars
- 82
- Forks
- 37
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 22
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from confluentinc/cli
-
Difficulty 2/5 1-3 hours Newbie friendliness 69/100
confluentinc/cli#3568 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
confluentinc/cli#3585 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 55/100
confluentinc/cli#3147 · 3 comments · 1 reaction ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 35/100
confluentinc/cli#3072 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 3/5 1-2 days Newbie friendliness 45/100
confluentinc/cli#3056 · 3 comments ·
Maintainers usually reply within 1 day
All issues in confluentinc/cli
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
-
bug triage/pending
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
core enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
forgelab-dev/lfa-forgenet#109 ·
Maintainers usually reply within 1 day
-
[mesheryctl] Enforce maximum positional argument bounds in system context view to prevent silent argument discardingPossibly taken @madhavzanwar claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
meshery/meshery#24055 · 1 comment ·
Maintainers usually reply within 1 day