Double free or corruption C++ Avro
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
Research direction
Start with the supplied main function and reproduce the failure around avro::GenericDatum in json2avro. Read examples/kafka-serdes-avro-console-producer.cpp and src/avro/fh_price_t.json to compare the expected Avro setup with this example. Done means identifying the ownership or lifetime issue and confirming the reproducer no longer reports double-free or corruption.
Written by the indexing model from the issue text.
Description
I need to serialize a JSON string. I used the example code
./examples/kafka-serdes-avro-console-producer.cpp
to write a main, which serializes a JSON string as follows.
{"s":"BTCUSDT","c":"spread","u":123456789,"a":20000.4,"b":21000.5,"A":1.25,"B":0.58,"p":-1,"P":-1,"st":-1,"rt":1675000000123000}
Here is the code
#include "fh_price_t.h"
#include "avro/Encoder.hh"
#include "avro/Decoder.hh"
#include <avro/Generic.hh>
#include <avro/Specific.hh>
#include <avro/Exception.hh>
#include "libserdes/serdescpp-avro.h"
#include <iostream>
#include <sstream>
#include <string>
#include <fstream>
#define FATAL(reason...) do { \
std::cerr << "% FATAL: " << reason << std::endl; \
exit(1); \
} while (0)
template<class T> std::string to_string_a (const T& x) {
std::ostringstream oss;
oss << x;
return oss.str();
}
/**
* Convert JSON to Avro Datum.
*
* Returns 0 on success or -1 on failure.
*/
int json2avro (Serdes::Schema *schema, const std::string &json,
avro::GenericDatum **datump) {
avro::ValidSchema *avro_schema = schema->object();
/* Input stream from json string */
std::istringstream iss(json);
auto json_is = avro::istreamInputStream(iss);
/* JSON decoder */
avro::DecoderPtr json_decoder = avro::jsonDecoder(*avro_schema);
avro::GenericDatum *datum = new avro::GenericDatum(*avro_schema);
try {
/* Decode JSON to Avro datum */
json_decoder->init(*json_is);
avro::decode(*json_decoder, *datum);
} catch (const avro::Exception &e) {
std::cerr << "% JSON to Avro transformation failed: "
<< e.what() << std::endl;
return -1;
}
*datump = datum;
}
int main (int argc, char* argv) {
izycoinstestavro::fh_price_t price{};
price.s = "BTCUSDT";
price.c = "spread";
price.u = 123456789;
price.a = 20000.4;
price.A = 1.25;
price.b = 21000.52;
price.B = 0.58;
price.p = -1;
price.P = -1;
price.st = -1;
price.rt = 1675000000123000;
std::ifstream file_ifstream("../src/avro/fh_price_t.json");
std::stringstream buffer;
buffer << file_ifstream.rdbuf();
std::string schema_json = buffer.str();
Serdes::Conf *sconf = Serdes::Conf::create();
Serdes::Schema *schema;
std::string errstr;
if (sconf->set("schema.registry.url", "http://localhost:8081", errstr))
FATAL("Conf failed: " << errstr);
if (sconf->set("serializer.framing", "cp1", errstr))
FATAL("Conf failed: " << errstr);
Serdes::Avro *serdes = Serdes::Avro::create(sconf, errstr);
if (!serdes)
FATAL("Failed to create Serdes handle: " << errstr);
schema = Serdes::Schema::add(serdes, "fh_price_t", schema_json, errstr);
if (!schema)
FATAL("Failed to register schema " << "fh_price_t" << ": " << errstr);
std::cout << "% Registered schema " << schema->name() << " with id " << schema->id() << std::endl;
avro::GenericDatum *datum = NULL;
std::vector<char> out;
/* Convert JSON to Avro object */
std::string line = to_string_a(price);
json2avro(schema, line, &datum);
//std::cout << to_string_a(price) << std::endl;
/*if (rr == -1) {
FATAL("Failed to convert JSON to Avro " << to_string_a(price) << ": " << errstr);
}
// Serialize Avro
if (serdes->serialize(schema, datum, out, errstr) == -1) {
std::cerr << "% Avro serialization failed: " << errstr << std::endl;
delete datum;
exit(1);
}
delete datum;
std::cout << "Data Size : " << out.size() << std::endl;*/
return 0;
}
When I run it, I get a double free or corruption (out) error. The error occurs at the output of the function. I have located the line that causes the error (because when I remove it, the error disappears)
avro::GenericDatum *datum = new avro::GenericDatum(*avro_schema);
I would like to know why and how to solve it.
- Dominant language
- C
- Stars
- 12
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from confluentinc/libserdes
-
Difficulty 3/5 Half a day Newbie friendliness 22/100
confluentinc/libserdes#68 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 15/100
confluentinc/libserdes#66 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
confluentinc/libserdes#64 ·
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
confluentinc/libserdes#63 · 2 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
confluentinc/libserdes#59 · 2 comments ·
All issues in confluentinc/libserdes
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
arduino/ArduinoCore-mbed#1126 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
-
security
Difficulty 1/5 Under an hour Newbie friendliness 93/100
btclib-org/btclib-secp256k1#1427 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
esmuellert/codediff.nvim#594 ·
-
Difficulty 1/5 1-3 hours Newbie friendliness 67/100