Event Tracking

When debugging complex issues, the existing logging and stats observability is often inadequate as it will often fail to incorporate enough information to understand what is going on.

To aid these debugging scenarios, an event tracking library would be very useful.

As a first step, something that looks very similar to logs could provide some immediate value:

EMIT_EVENT(dns_resolution_complete, “dns_name”, name_);

if (EVENT_ENABLED(dns_resolution_failure)) {
  // ... do expensive calculation
  EMIT_EVENT(dns_resolution_failure, info); 
}

This would emit events very similar to how we emit logs today, but provide more stable identifiers instead of free form strings that might change as code is refactored. The idea of using an event name also allows for toggling of individual events, which provides more coarse grained flexibility compared to logging.

The events would be propagated to a sink similar to how logs are, using an interface that may look something like

void onEvent(absl::string_view name, std::initializer_list<std::string> info);

Future iterations of this could include adding support for scopes, potentially via ScopeTrackedObject or via other event specific scopes. These scopes could then also provide timings, powering a lot of perf related debugging (e.g. https://github.com/envoyproxy/envoy/issues/16988)

A strawman API with support for scopes to get things going:

struct EventContext {
  const ScopeTrackedContext* context_;
  std::vector<EventScope> event_scopes_;
};

class EventSink {
  virtual void onEvent(absl::string_view name, const std::vector<std::string>& info, const EventContext& context) PURE;
  virtual void onScopeStarted(absl::string_view name, const std::vector<std::string>& info, const EventContext& context, uint64_t scope_id) PURE;
  virtual void onScopeEnded(absl::string_view name, const std::vector<std::string>& info, const EventContext& context, uint64_t scope_iD) PURE;
};

auto scope_handle_ptr = START_EVENT_SCOPE("dns_resolution", "host", address);
EMIT_EVENT("dns_failure", "reason", reason";

@rojkov @jmarantz @mattklein123

Happy to bring this to a Google doc, but figured I'd gather some immediate feedback before writing up a more complete proposal.