Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

hotlane runs 10-year backlog enrichment on a 1.0s tick (--enrich-since-hours 87600), so it never idles — policy call needed

Open
  • Is something wrong?
#738 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
python, sqlite
Domain
ai, databases, performance

Research direction

Start by reviewing the hotlane launch configuration, especially --enrich-since-hours, --interval, and the related context in issues #735 and #737. Done means a maintainer rules on ownership of historical enrichment, then the selected behavior is implemented and the no-recent-writes CPU usage is measured with ps.

Written by the indexing model from the issue text.

Description

Problem — a policy question, not a bug report

com.brainlayer.hotlane-brainbar runs with:

--interval 1.0 --recent-limit 5 --backlog-interval 10.0 --backlog-batch 4
--enrich-interval 10.0 --enrich-limit 4 --enrich-since-hours 87600

87600 hours = 10 years. So the hot lane is also the backlog lane: every 10s it looks for
anything unenriched across all history and runs enrichment on it, and on a 1M-row DB there is
essentially always something unenriched. The daemon therefore never becomes idle by construction.

Measured 2026-08-24: 120% CPU sustained for its whole 29-minute life. sample shows the busy
threads in SGEMM / APL_sgemm (libBLAS), at::native::vectorized_gelu (libtorch_cpu) and
tokenizers rayon workers — i.e. bge embedding inference running continuously, never sleeping.
That is the arithmetic consequence of those two flags, not a defect in the loop body.

Why PR #735 does not close this

#735 adds idle backoff (idle cycles back off geometrically to a 30s cap; any real work snaps the
cadence back to interval). That fixes a genuinely-idle hotlane re-scanning a 14.8 GB DB once per
second. It cannot help here, because with a 10-year lookback the lane is never idle — it always
finds work, so it always resets to the 1.0s cadence.

The actual decision needed

Should the hot lane do historical backlog enrichment at all, or should that belong to the dedicated
backlog/enrichment path with the hot lane bounded to genuinely recent writes?

This changes enrichment coverage, so it is Etan's call, not a unilateral code change. Flagging
rather than "fixing" for that reason.

Options, roughly in increasing intrusiveness:

  1. Leave as-is and accept a permanently-busy core.
  2. Reduce --enrich-since-hours on the hot lane and let the backlog lane own history.
  3. Keep the 10-year sweep but move it off the 1.0s tick onto its own much slower schedule.

Acceptance criteria

  • A ruling on which lane owns historical enrichment.
  • Whatever is chosen, the hot lane's steady-state CPU when there are no recent writes is bounded and
    measured with ps, not asserted.

Related: #735 (idle backoff), #737 (deploy drift).

— brainlayerClaude (lead) · claude-code/claude-opus-5

Dominant language
Python
Stars
9
Forks
7
Avg merge
4h 43m
Merged PRs (30d)
360

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from EtanHey/brainlayer

All issues in EtanHey/brainlayer

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.