grafana/tempo

Support service graph edges that are not strictly `client` -> `server`

Open

#1,418 opened on May 3, 2022

View on GitHub
 (2 comments) (3 reactions) (0 assignees)Go (727 forks)auto 404
Priority Lowcomponent/metrics-generatorhelp wantedkeepalive

Repository metrics

Stars
 (5,370 stars)
PR merge metrics
 (PR metrics pending)

Description

Is your feature request related to a problem? Please describe.

We use istio with the istio ingress gateway which initiates traces from incoming requests, or adds a span in case trace information is already present. This span always has span.kind = client, which makes sense if the ingress gateway initializes the trace. However, if we have started a trace already from a different cluster when arriving at the ingress we will get two spans in a row with kind client. This becomes an issue due to the requirement of strict client -> server edges when building service graphs using the metric-generator this prevents us from building inter-cluster service graphs. Example:

span.kind    client               client              server
service        A     ->    istio-ingressgateway    ->   B

The edge between service A and istio-ingressgateway will never be completed and we get very flat service graphs as a result.

Describe the solution you'd like

A relaxation of the span kind requirement between the two nodes of an edge, either by default of via a feature toggle. Ideally istio/envoy should probably generate two spans, first a server and then a client kind, but that also seems a bit wasteful if the only point is to set a different span.kind.

Contributor guide