jaegertracing/jaeger

Integration with external logs storage when displaying traces

Open

#649 建立於 2018年1月13日

在 GitHub 查看
 (21 留言) (39 反應) (0 負責人)Go (18,974 star) (2,326 fork)batch import
enhancementhelp wanted

描述

Many organizations have log aggregation setups, such as shipping all logs to Elasticsearch. For logs corresponding to traced transactions, it may not be feasible or desired to direct them all to the tracing backend. As long as the logs are labeled with trace/span IDs, it is possible to reassemble the logs from the logging backend when viewing traces in Jaeger UI.

This ticket is to track the design and implementation details of such solution.

The high level design is something like this:

+--------------+        +--------------+         +--------------+                    
|              |        |              |         |              |                    
|  Jaeger UI   |------->| jaeger-query |-------->|  logs query  |                    
|              |        |              |         |              |                    
+--------------+        +--------------+         +--------------+                    
                                                        |                            
                                     +------------------+--------------------+       
                                     |                  |                    |       
                             +--------------+    +--------------+    +--------------+
                             | logs storage |    | logs storage |    | logs storage |
                             | Elasticsearch     |   Kafka      |    |    HDFS      |
                             |              |    |              |    |              |
                             +--------------+    +--------------+    +--------------+

The log query service is meant to serve as an abstraction of different log storage types.

Open questions

  • what is the API of the log-query service
  • does jaeger-query retrieve logs when returning the trace to the UI or on-demand per-span

貢獻者指南

技術棧
goelasticsearch
領域
observabilitybackendapi
議題類型
feature
難度面向新貢獻者的預計實作難度，1 表示很小改動，5 表示專家級工作。
4
預計時間有經驗貢獻者完成調查、實作、測試並準備 pull request 的粗略時間範圍。
over 1 week
活動狀態議題目前的可參與程度：新鮮、活躍、陳舊、阻塞或等待維護者輸入。
stale
清晰度議題是否清楚說明預期改動、驗收標準和下一步。
needs investigation
前置要求
Knowledge of distributed tracingFamiliarity with Jaeger architecture
新手友善度1-100 的估計分數，表示該議題對首次貢獻者的友善程度。
15
研究方向
The issue outlines a high level design for integrating external log storage with Jaeger UI. Key open questions include the API of a proposed log query service and whether logs should be fetched per span or on demand per trace. To move forward, investigate existing jaeger query service internals (e.g., in ./cmd/query/) and the Jaeger UI code (in ./jaeger ui/) to understand how data is currently fetched. Explore existing log aggregation solutions like Elasticsearch and Kafka to define the log query API. Review comments in the issue for additional context and potential design constraints.