Cache results of aggregation / histogram requests per sealed fraction

Open
#491 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Quiet
Tech stack
elasticsearch, go, redis
Domain
databases, performance

Research direction

Start with docs/en/11-benchmarks.md and the existing cache subsystem to understand the aggregation and histogram read path and the stated performance gap. Compare that with the seq-ui logs_lifespan/events pattern; the issue is complete only after maintainers decide whether caching belongs in seq-db or seq-ui and define the accepted scope.

Written by the indexing model from the issue text.

Description

Dashboards re-issue the same aggregation/histogram queries every N seconds,
and seq-db recomputes them from scratch on every refresh — the most
expensive read path (docs/en/11-benchmarks.md) with the most repetitive
workload. Caching the results seems natural (cf. Elasticsearch's shard
request cache), and we'd like to know where the maintainers see it living:

  1. In seq-db (store level, per sealed fraction). Sealed fractions are
    immutable, so a result fully covering a fraction can be cached without
    TTL and stays exact; sliding windows reuse interior fractions. We built
    a prototype this way: an opt-in aggs layer in the existing cache
    subsystem, hit ≈ 2.7µs vs 14ms recompute on a 200k-doc fraction. Happy
    to submit a PR if this direction is acceptable.

  2. In seq-ui (Redis, TTL). Simpler, follows the existing
    logs_lifespan/events pattern, but results go stale within the TTL,
    a shifted time window is always a full miss, and other API consumers
    don't benefit.

Is per-fraction caching in seq-db something you'd accept, or do you consider
result caching out of scope for the engine and better kept in seq-ui?

Dominant language
Go
Stars
131
Forks
16
Avg merge
2d 4h
Merged PRs (30d)
11

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ozontech/seq-db

All issues in ozontech/seq-db

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.