apache/pinot

Reduce the amount of trace info

Open

#9,523 创建于 2022年10月3日

在 GitHub 查看
 (2 评论) (0 反应) (1 负责人)Java (1,234 fork)batch import
good first issuehelp wanted

仓库指标

Star
 (4,937 star)
PR 合并指标
 (平均合并 6天 7小时) (30 天内合并 186 个 PR)

描述

Currently the default tracer will print an entry per block (up to 10K docs). For a query hitting 1B docs, it will log 100K entries, which is not useful for debugging. It can even hang the controller UI when sending a query with trace on.

We should reduce the tracing info by merging the same entry type into one entry for each segment and just print the total time per entry type.

Other things to consider:

  • Should we merge the same entry type across segments? Each query can still hit thousands of segment per server
  • We can consider keeping a global cross segments aggregated trace info, then keep 10 segment level aggregated trace if the query hits too many segments

贡献者指南