jaegertracing/jaeger

Find Traces Matching Multiple Services

Open

#423 opened on Sep 24, 2017

View on GitHub
 (8 comments) (17 reactions) (0 assignees)Go (18,974 stars) (2,326 forks)batch import
help wantedui

Description

Counterpart to UI ticket https://github.com/jaegertracing/jaeger-ui/issues/180

Currently, you can only search spans belonging to a one service. It would be nice if you could find traces belonging to multiple services. Then, you could search tags where error:true on all tags or search for traces which include multiple services.

Implementation plan:

  • define a new Storage Capability property indicating if multi-service search is supported (default: not supported)
  • change the UI to receive this flag via existing capabilities mechanism and allow / disallow not specifying service name in the search panel
  • implement multi-service search in memory store (should be trivial)
  • implement multi-service search in Elasticsearch / Opensearch (probably the easiest across real backends)
  • try to implement it in Badger (difficulty: unknown)
  • for Cassandra, introduce another flag to allow / disallow multi-service searches. The motivation is that with the current indexing scheme the only way to support multi-service search is by reading all service names and iterating across all of them (which I think what Zipkin implementation did). This might work for small deployments, but would be useless for large companies like Uber, so they would likely opt to disable it.
    • Another possibility is to use secondary index in Cassandra. The current schema is not designed for that, so difficulty: unknown.

Contributor guide

Tech stack
goelasticsearch
Domain
backendobservability
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Go programmingJaeger storage backendsElasticsearch basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
25
Research direction
The implementation plan outlines steps: define a StorageCapability, modify UI to use the flag, implement multi service search in memory store, Elasticsearch, Badger, and Cassandra. The main codebase is in Go, storage backends are in separate packages. The UI changes are in jaeger ui repo (linked in #180). Start by understanding the storage capability interface in the Go code, then examine memory store tests to add multi service search. For Elasticsearch, look at the storage implementation. Consult comments in the issue for any ongoing discussion.