[Feature] Implement batch TopN aggregation for historical data on non-hot nodes
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 30/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- java
- Domain
- observability-sre
Research direction
The issue names no files, tests, or entry points. Start by reading related issue #13285 and locating the existing TopN aggregation and warm/cold storage workflows; done should include configurable batch tasks, historical time ranges, naming conventions, and post-import aggregation.
Written by the indexing model from the issue text.
Description
Search before asking
- I had searched in the issues and found no similar feature requirement.
Description
Related to #13285
Implement a batch processing system that enables TopN aggregation for historical data stored on warm and cold nodes. This feature will support scenarios where historical data is imported and users need to generate TopN aggregations retroactively with flexible configuration options.
Background
Non-hot storage tiers (warm and cold nodes) often contain historical data that may be imported in bulk. Currently, there's no efficient mechanism to perform TopN aggregations on this historical data without impacting real-time operations or requiring manual intervention.
Proposed Features
Develop a batch processing capability that supports:
- On-demand TopN aggregation through configurable batch tasks
- Flexible time-range specifications for historical data processing
- User-defined naming conventions for generated TopN results
- Support for importing historical data with subsequent batch aggregation processing
Use case
No response
Related issues
No response
Are you willing to submit a pull request to implement this on your own?
- Yes I am willing to submit a pull request on my own!
Code of Conduct
- I agree to follow this project's Code of Conduct
- Dominant language
- Java
- Stars
- 25k
- Forks
- 6.6k
- Avg merge
- 12h 1m
- Merged PRs (30d)
- 43
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/skywalking
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/skywalking#14239 ·
Maintainers usually reply within 1 day
-
AI Assistant database enhancement
Difficulty 5/5 Over a week Newbie friendliness 24/100
apache/skywalking#14237 ·
Maintainers usually reply within 1 day
-
database enhancement
Difficulty 5/5 Over a week Newbie friendliness 25/100
apache/skywalking#14236 ·
Maintainers usually reply within 1 day
-
AI Assistant database enhancement
Difficulty 5/5 Over a week Newbie friendliness 25/100
apache/skywalking#14235 ·
Maintainers usually reply within 1 day
-
database enhancement
Difficulty 5/5 Over a week Newbie friendliness 30/100
apache/skywalking#14234 ·
Maintainers usually reply within 1 day
All issues in apache/skywalking
Similar issues
-
Re-executed query with IN condition on an index column returns deleted rows (stale IndexCursor)Possibly taken @dbusche claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
h2database/h2database#4580 ·
Maintainers usually reply within 1 day
-
vips-drift
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Stirling-Tools/JPDFium#300 ·
Maintainers usually reply within 1 day
-
hermes-queue
Difficulty 1/5 1-3 hours Newbie friendliness 85/100
chrisalunlloyd2-sudo/mindpalace#403 ·
Maintainers usually reply within 1 day
-
Guidelines: document how to build a use case against a local, unpublished Flow/browserless-test changePossibly taken @totally-not-ai claimed this today. Open
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
vaadin/use-cases#452 · 1 comment ·
Maintainers usually reply within 1 day
-
area/storage area/storage/sql
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Apicurio/apicurio-registry#11159 ·
Maintainers usually reply within 1 day