[Feature] Trace Pipeline Support in BanyanDB
维护者通常 1 天内回复
@hanahmily 已经在做这个了。
开始于 2025年12月25日。
评估
这个 Issue 还没有评估数据。
描述
Search before asking
- I had searched in the issues and found no similar feature requirement.
Description
🌟 Feature Request: Trace Pipeline Support in BanyanDB (Target: v0.11)
Summary
Introduce a trace pipeline framework into BanyanDB to enable trace-oriented processing, sampling, and enrichment directly within the database layer.
The pipeline will serve as the foundation for intelligent, context-aware trace handling—allowing dynamic update, analysis, and retention decisions while preserving schema and performance guarantees.
Background
BanyanDB currently supports efficient ingestion and querying of trace data, but processing is span-oriented and lacks capabilities for post-ingestion, trace-level operations.
Modern observability scenarios—such as tail-based sampling, cross-service trace enrichment, or behavioral trace analysis—require the database to treat a group of spans (sharing the same trace ID) as one logical unit.
Introducing a trace pipeline provides a structured mechanism to:
- Analyze and modify trace data after ingestion.
- Aggregate spans consistently, regardless of their reported order or source.
- Make smarter data retention decisions based on complete trace context.
Goal
Deliver a trace-level pipeline mechanism that:
- Processes collected spans as unified trace entities in best-effort strategy(No 100% guaranteed).
- Supports runtime enrichment and analysis of existing tag values, further considering to declear trace-level schema, e.g. trace latency.
- Maintains persistent file-based processor context across ingestion batches for continuity and scale.
- Enables tail-based sampling via a reserved deletion/retention flag.
- Integrates cleanly with OAP for schema declaration, dynamic loading, and test validation.
Use case
This issue includes several key features and use cases, mostly around trace post-analysis, e.g., tail sampling.
Most things were discussed between @hanahmily and me at a high level. No detail is confirmed yet.
Related issues
No response
Are you willing to submit a pull request to implement this on your own?
- Yes I am willing to submit a pull request on my own!
Code of Conduct
- I agree to follow this project's Code of Conduct
- 主要语言
- Java
- 星标
- 25k
- 派生
- 6.6k
- 平均合并
- 15 小时 37 分钟
- 30 天内合并 PR
- 32
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
apache/skywalking 的其他 Issue
-
[Bug] BanyanDB trace block metadata format changed in 0.11 without a file format version bump可能已有人在做 @hanahmily 今天认领。 未关闭bug database
apache/skywalking#14169 · 已指派 1 人 ·
维护者通常 1 天内回复
-
chore CLI database
难度 5/5 一周以上 新手友好度 25/100
apache/skywalking#14168 ·
维护者通常 1 天内回复
-
[Bug] BanyanDB stream scans run out of query memory even with a small limit可能已有人在做 @hanahmily 于 3 天前认领。 未关闭bug database
apache/skywalking#14165 · 3 条评论 · 已指派 2 人 ·
维护者通常 1 天内回复
-
[Feature] Canopy auth 2/2: SSO (OIDC/OAuth2), break-glass and the auth design doc可能已有人在做 @hanahmily 于 2 天前认领。 未关闭database feature
难度 5/5 一周以上 新手友好度 20/100
apache/skywalking#14162 ·
维护者通常 1 天内回复
-
[Feature] Canopy auth 1/2: local auth — auth config, argon2id login, server-side sessions, API tokens可能已有人在做 @hanahmily 于 3 天前认领。 未关闭database feature
难度 5/5 一周以上 新手友好度 35/100
apache/skywalking#14159 ·
维护者通常 1 天内回复
查看 apache/skywalking 的全部 Issue
相似的 Issue
-
Bug
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 1 天内回复
-
protobuf
难度 2/5 1-3 小时 新手友好度 84/100
FasterXML/jackson-dataformats-binary#874 ·
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 75/100
-
难度 1/5 1 小时以内 新手友好度 85/100
-
难度 2/5 1-3 小时 新手友好度 62/100
HexaHexes/infinite-dungeon-spawn-safety#17 ·
维护者通常 1 天内回复