Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

watcher: event-driven discovery (FSEvents) — polling measured out at ~2.4s/poll, gate needs <5%

Open
  • Is something wrong?
#760 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
macos, python
Domain
operating-systems, performance, tooling

Research direction

Start by locating the watcher entry point around poll_once and _discover_jsonl_files(), then review the existing (mtime, size) skip gate and prune_missing_files retry logic from #759. Design the FSEvents wake-up path with a slow reconciliation sweep and preserve the documented refusal conditions. Done means three externally sampled, 10-minute caught-up idle soaks all remain below 5% CPU.

Written by the indexing model from the issue text.

Description

Why this exists

R3 (#759) fixed the two defects responsible for the JSONL watcher's idle CPU burn and took
poll_once from 18.0s → 2.4s (7.4×). It still does not clear the <5% idle-CPU gate,
and com.brainlayer.watch remains => disabled.

This issue records exactly how far polling can go, so whoever picks it up starts from evidence
rather than re-deriving it. Everything below was measured on the M4 against the real corpus
(~12,800 JSONL files, 21,529 registry entries), nice -n 15 taskpolicy -b.

The polling floor, measured

stage poll_once, steady state duty @30s
before R3 18.0s 60%
after prune-retry fix 5.0s 17%
after denylist memo (shipped in #759) 2.4s 8.3%

Of that remaining 2.4s, discovery is 2.19–2.56s — a **/*.jsonl walk across 5 provider roots
plus a stat on every file. A standalone bare glob+stat sweep of the same corpus measures
~1.03s, so ~1s is the hard floor of any polling design here. There is no further meaningful
win left inside the poll loop.

Soak results (10 min, sampled every 10s with ps, externally — never the watcher's own health file)

run code mean idle CPU verdict
1 pre-change-detector 4.88% pass by 0.12
2 shipping (#759) 6.41% FAIL

Near-identical code, same method, same machine — 1.5 points apart. max ≈ 98.6% in both,
because the cost is a ~2.4s burst per poll, not a smooth trickle.

The load-bearing conclusion: run-to-run variance is as large as the margin. A single 10-minute
soak cannot resolve a 5% gate on a design that sits at 4–6.5%. Re-running until one passes is
selection, not evidence.

Why a longer poll interval is the wrong fix

Raising the interval divides the floor by a bigger number. It buys margin with ingestion latency
and still leaves the burst — and at the variance above it does not buy reliable margin.
--poll 60 was tried during R3 and the shipping code still measured 6.41%.

What to build

Event-driven discovery via FSEvents — the first branch of the original R3 constraint
("event-driven, or batched at ≥30 s"), which the polling branch has now been measured to the end
of. The watcher should be woken by the filesystem instead of asking it ~2,880 times a day whether
anything changed.

Design notes from what R3 touched:

  • Discovery and read are already separable. _discover_jsonl_files() finds new files; the
    (mtime, size) skip gate added in #759 already makes already-tracked, unchanged files free. An
    FSEvents stream supplies exactly the "what changed" signal the sweep is currently brute-forcing.
  • Keep the skip gate's refusal conditions. They exist because a false skip stalls a session
    silently: no tailer yet, tailer.offset < size (a read capped by max_lines_per_file leaves the
    stat unchanged but the tailer short), a complete line still buffered, pending
    quarantine/ingestion failures. Any event-driven path needs the same guards on its fallback sweep.
  • Keep a slow full sweep as a backstop. FSEvents can drop events (volume unmount, coalescing,
    historical replay gaps). A periodic reconciliation sweep — minutes, not seconds — is the safety
    net, and at that cadence its ~2.4s cost is negligible.
  • The prune interacts. prune_missing_files is now retried on a parent-dir change-detector plus
    a 900s timer (BRAINLAYER_WATCHER_OFFSET_PRUNE_RETRY_S). It is deliberately unable to prune
    entries whose root is unmounted; do not "fix" that guard.

Acceptance

  • 10-minute idle soak repeated at least 3 times, mean idle CPU <5% in every run — one run
    is not a measurement at this variance.
  • Sampled externally (ps), never from the watcher's own health file: a soak the watcher grades
    itself on inherits whatever probe defects it has.
  • Measured from a caught-up offset registry, so it is idle cost and not backlog catch-up.
    Reporting catch-up CPU as idle CPU is the specific false green R3 hit twice.

Not in scope here

com.brainlayer.watch re-enable is a separate decision currently with Etan: accept ~5–6.5% and
enable now, leave it down until this lands, or something else. This issue is the engineering
path, not that decision.

Related: #759.

— brainlayerClaude-e358a78e (worker) · claude-code/claude-opus-5

Dominant language
Python
Stars
9
Forks
7
Avg merge
2h 8m
Merged PRs (30d)
225

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from EtanHey/brainlayer

All issues in EtanHey/brainlayer

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.