avro::decode not able to handle NULL array. Thows "Incorrect token in the stream. Expected: Array start, found Null"

Open
#49 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
cpp
Domain
backend

Research direction

Start at the Json2Avro entry point and inspect how avro::jsonDecoder and avro::decode process list2 when the JSON value is null. Determine the expected behavior for nullable arrays and records, then verify that valid null values no longer fail while existing non-null decoding still works.

Written by the indexing model from the issue text.

Description

The serdes CPP code is not able to handle NULL arrays when converting json to avro.

sample excerpt of input json that fails:
,"list1":[{"test1", "test2"}],"list2":null,"listItemCount":0,.........

It bails out at "avro::decode(*json_decoder, *datum);" while handling list2. How can we make decode accept null values which could be valid in some cases. Same question applies to a record that could be null too. Thanks.

static int
Json2Avro(Serdes::Schema *schema, const std::string &json, avro::GenericDatum **datump)
{
avro::ValidSchema *avro_schema = schema->object();

std::istringstream iss(json);
auto json_is = avro::istreamInputStream(iss);
avro::DecoderPtr json_decoder = avro::jsonDecoder(*avro_schema);
avro::GenericDatum *datum = new avro::GenericDatum(*avro_schema);

try {
    json_decoder->init(*json_is);
    avro::decode(*json_decoder, *datum);
}
catch (const avro::Exception &ex) {
    cerr << "JSON to avro transformation failed: " << ex.what() << endl;
    return -1;
}

*datump = datum;

return 0;

}

Dominant language
C
Stars
12
Forks
62
PR merge metrics
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from confluentinc/libserdes

All issues in confluentinc/libserdes

Similar issues

More C issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.