envoyproxy/envoy

Host metadata take up a lot of memory

Open

#9.532 geöffnet am 2. Jan. 2020

Auf GitHub ansehen
 (13 Kommentare) (0 Reaktionen) (1 zugewiesene Person)C++ (5.373 Forks)batch import
area/perfarea/xdshelp wanted

Repository-Metriken

Stars
 (27.997 Stars)
PR-Merge-Metriken
 (Durchschn. Merge 8T) (378 gemergte PRs in 30 T)

Beschreibung

Title: Host metadata take up a lot of memory

Description:

Currently, our host metadata use map<string, google.protobuf.struct > type, because of the use of google.protobuf.struct, which contains a lot of type reflection information, which takes up more memory space. Even without any metadata information, an empty object would take up 144 bytes.

The following code is my test


message Metadata {
  // Key is the reverse DNS filter name, e.g. com.acme.widget. The envoy.*
  // namespace is reserved for Envoy's built-in filters.
  map<string, google.protobuf.Struct> filter_metadata = 1;
}

  tutorial::Metadata meta;
  google::protobuf::Struct struct_obj;
  auto& fields_map = *struct_obj.mutable_fields();
  fields_map["test_key"] = stringValue("test_value");
  google::protobuf::Struct struct_inner;
  (*struct_inner.mutable_fields())["inner_key"] = stringValue("inner_value");
  google::protobuf::Value val;
  *val.mutable_struct_value() = struct_inner;
  fields_map["test_obj"] = val;
  (*meta.mutable_filter_metadata())["com.test"] = struct_obj;

  std::cout << meta.SpaceUsedLong() << std::endl;

  tutorial::Metadata meta1;
  std::cout << "empty:" << meta1.SpaceUsedLong() << std::endl;

Output:

720
empty:144

Is it possible to use a Map instead of a Struct, or to use Any nested Struct instead? In addition, we can actually optimize the default values so that we don't have to build Metadata objects when we don't have any metadata information.

@mattklein123

Contributor Guide