HermitCrab synthesis: 25.6% of synthesis steps end in SurfaceFormMismatch; early-reject check not built (research lead)

未关闭
  • 哪里不对？
#603 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
45/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
csharp
领域
performance

调研方向

Start with docs/hermitcrab-2026-10-27-synthesis-to-analysis-round.md, especially lines 96 onward and the cited lines 496 and 679; the issue points to branch docs/hc-optimization-ledger at 2eba2f50. Measure work steps per grammar for synthesis candidates that end in SurfaceFormMismatch, and compare their share with the existing census. Done means reporting whether the share is material enough to justify building the proposed count-only check.

由索引模型根据 Issue 内容生成。

描述

Claude-drafted.

25.6% of synthesis steps on the five real-grammar samples end in SurfaceFormMismatch, after the whole rule stack has run. Whether an intermediate-shape check could reject those candidates earlier is not measured.

Status: research lead, not built. There is no soundness argument yet.

What we know (C# hc.dll, traced samples, unit: synthesis steps and analysis events):

  • SurfaceFormMismatch is 25.6% of synthesis steps pooled (279,872 steps); 37.9% on Indonesian, 27.2% on Mbugwe, 16.9% on Amharic.
  • Synthesis is a small part of total work (synthesis steps plus analysis events): 1.0% Aweti, 1.8% Sena, 9.0% Mbugwe, 9.4% Indonesian, 14.7% Amharic. A perfect check would therefore remove well under 25.6% of total work (inferred).
  • A check at each stratum boundary has no soundness argument: analysis shapes over-approximate (optional and underspecified nodes), and the earlier synthesis-side length bound (ledger row 8) lost valid parses.

Not known: how many SurfaceFormMismatch steps a sound check would cut, and whether any successful candidate would fail it.

Proposed change: count, do not prune. At each stratum boundary, record whether the check would fire and whether any successful candidate fails it, in steps per grammar. Build the check only if the share is material. Kill it if any success fails it on any corpus.

Related: #560 (E1, open). Retired record: PR #490 (closed). PanGloss: sillsdev/PanGloss#105 (pointer, open).

Next: run the boundary census on hc.dll and post steps cut per grammar plus any success that fails the check.

Evidence -- where the numbers come from

The census harness is test-only (branch research/failure-reason-stats at af97348b, master c2454760 with PR #480 merged; whether the branch still exists was not checked). A step is one rule-level synthesis trace event. Counts come from traced parses, which turn analysis merging off, so more analyses are explored than in an untraced run. 22.1% of steps have no terminal event and are unattributed.

Corpus (sample) SurfaceFormMismatch, % of synthesis steps Synthesis, % of total work
Pooled real grammars 25.6%
Aweti, every 2nd word 1.0%
Sena, every 100th 1.8%
Mbugwe, every 40th 27.2% 9.0%
Indonesian, all 70 37.9% 9.4%
Amharic, first 28 16.9% 14.7%

Blank cells were not given in the record. Conformance suite: 6.9% of its steps, 22.9% synthesis share.

Sources -- pinned to 2eba2f50 (closed PR #490 record)
What would reopen this

A soundness argument for a stratum-boundary check, or a census where no success fails it and the cut share is material.

主要语言
C#
星标
29
派生
16
平均合并
6 天 7 小时
30 天内合并 PR
14

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 有 Pull Request 模板
  • 没有贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

sillsdev/machine 的其他 Issue

查看 sillsdev/machine 的全部 Issue

相似的 Issue

更多 C# Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。