Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

PERFORMANCE.md states a 46% observer overhead that measures 0%, and the gate that should have caught it is one-sided by construction

未关闭
  • 哪里不对？
#1,206 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
65/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
活跃
技术栈
c, shell
领域
build-system, documentation, performance, testing-qa

调研方向

Start by reading docs/PERFORMANCE.md:138-140, bench/check_regression.sh, and bench/baseline.txt, then rerun the observer benchmark with n=5 Ir measurements. Check the observer-gate history referenced by #915 and PR #1034. Done means the current cost is established, improvements cannot silently pass the gate, and the documented figures are mechanically checked or generated from the baseline.

由索引模型根据 Issue 内容生成。

描述

area:docs area:gates found-by:critic kind:gate-defect

Found by re-running a number the docs state in the present tense.

docs/PERFORMANCE.md:139 documents observer overhead as "~46% more (Ir
≈ 93.2M vs 63.8M)" and :138 as ~28% slower wall-clock.

Measured

Reported by Jon, then reproduced independently here (callgrind, this dev
box, current main build):

documented Jon reproduced here
observed_loop Ir 93,231,983 59,567,345 59,519,029 (−36.2%)
unobserved_loop Ir 63,843,716 59,573,261 59,525,213 (−6.8%)
observed − unobserved +46% −5,916 −6,184 (−0.01%)
wall 37 vs 29 ms 6 vs 6 ms (median of 15)

The observed loop now runs fewer instructions than the unobserved one.
That is noise around zero — not a win in the other direction — but the
documented cost of observation has gone from 46% to nothing, and the docs
still assert it in the present tense with a deterministic Ir figure
attached, which is the form most likely to be believed.

The two documented figures are byte-identical to bench/baseline.txt
(observed_loop 93231983, unobserved_loop 63843716), so this is one
staleness event surfacing in two places — the same shape as the
bench/baseline.txt staleness found this morning.

The gate cannot catch this, and that is the more useful half

bench/check_regression.sh compares one direction only:

inc=$(( (cur - base) * 100 / base ))
if [ "$inc" -gt "$THRESHOLD_PCT" ]; then   # THRESHOLD_PCT=5

A drop of any size prints ok. observed_loop currently reads
-36% vs baseline and the gate is green.

So a one-sided gate cannot distinguish a real 36% improvement from a
36%-wrong baseline — both render identically, as good news. That is why
one unrecorded regeneration event desynced three artifacts at once and
nothing spoke: the baseline, the gate's effective threshold, and the
published docs.

docs/PERFORMANCE.md:140 compounds it by claiming the win is "now
regression-gated in both directions". It is gated in one.

What to do (measurement pass, not a quick edit)

  1. Establish what the observer costs now, properly: n=5 vs n=5, and
    Ir rather than wall (Ir here was stable; the wall figures are 6 ms
    and unusable at this size). If it is genuinely ~0, find out when it
    went to zero — the #915 observer gate (PR #1034) is the obvious
    candidate and would mean unobserved: no longer buys anything on this
    workload, which is a real documentation change, not a number swap.
  2. Make the gate two-sided. An improvement beyond the threshold should
    require the baseline to be re-pinned deliberately, exactly as a
    regression does. A silent improvement is the signature of a stale
    baseline far more often than of a win.
  3. Make line 139 derived. check_regression.sh --update regenerates
    the baseline; nothing regenerates the docs, so the copy in
    PERFORMANCE.md will rot again on the next legitimate update. Either
    generate that table from bench/baseline.txt, or enrol both figures in
    tools/docs_claims_check.sh so the doc fails when it disagrees with
    the file it was copied from.

Note on what is NOT wrong here

The section immediately above (lines 100–120) is careful work: thresholds
placed from measured spreads, false-red rates counted across 14 and 5
runs, a --selftest that proves the check can fail. It was built to be
re-run. Lines 121–139 are a table from "a 2020-era x86-64 Linux dev box"
that can only be re-run by someone who thinks to.

That is the distinction worth keeping: prose lessons age well, measured
claims rot silently — and they rot in whatever copies were made of them.
Same point as the stale CLAUDE.md passage in ouroboros, now with a second
instance and a mechanical cause.

主要语言
C
星标
3
派生
7
平均合并
3 小时 58 分钟
30 天内合并 PR
105

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

InauguralSystems/EigenScript 的其他 Issue

查看 InauguralSystems/EigenScript 的全部 Issue

相似的 Issue

更多 C Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。