envoyproxy/envoy

Host metadata take up a lot of memory

Open

#9532 aperta il 2 gen 2020

Vedi su GitHub
 (13 commenti) (0 reazioni) (1 assegnatario)C++ (5373 fork)batch import
area/perfarea/xdshelp wanted

Metriche repository

Star
 (27.997 star)
Metriche merge PR
 (Merge medio 8g) (378 PR mergiate in 30 g)

Descrizione

Title: Host metadata take up a lot of memory

Description:

Currently, our host metadata use map<string, google.protobuf.struct > type, because of the use of google.protobuf.struct, which contains a lot of type reflection information, which takes up more memory space. Even without any metadata information, an empty object would take up 144 bytes.

The following code is my test


message Metadata {
  // Key is the reverse DNS filter name, e.g. com.acme.widget. The envoy.*
  // namespace is reserved for Envoy's built-in filters.
  map<string, google.protobuf.Struct> filter_metadata = 1;
}

  tutorial::Metadata meta;
  google::protobuf::Struct struct_obj;
  auto& fields_map = *struct_obj.mutable_fields();
  fields_map["test_key"] = stringValue("test_value");
  google::protobuf::Struct struct_inner;
  (*struct_inner.mutable_fields())["inner_key"] = stringValue("inner_value");
  google::protobuf::Value val;
  *val.mutable_struct_value() = struct_inner;
  fields_map["test_obj"] = val;
  (*meta.mutable_filter_metadata())["com.test"] = struct_obj;

  std::cout << meta.SpaceUsedLong() << std::endl;

  tutorial::Metadata meta1;
  std::cout << "empty:" << meta1.SpaceUsedLong() << std::endl;

Output:

720
empty:144

Is it possible to use a Map instead of a Struct, or to use Any nested Struct instead? In addition, we can actually optimize the default values so that we don't have to build Metadata objects when we don't have any metadata information.

@mattklein123

Guida contributor