Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

HermitCrab conformance: verify blocked_by labels against the trace in the dead-rule gate

Open
  • Is something wrong?
#552 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
52/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
csharp
Domain
testing

Research direction

Start with CoverageReport.cs:113-114 to see how blocked_by labels enter the exercised-rule set, then read FailureRuleAttributor.cs:66 and its trace attribution behavior. Add a reviewed exemption for failures the trace cannot attribute, including zimed, and make a mutated label fail the gate. The issue points to code at #480 commit b49433d7.

Written by the indexing model from the issue text.

Description

Hermit Crab

Summary

The conformance suite's dead-rule gate credits a rule as exercised whenever an expected-failure word names it in blocked_by. It never checks that the rule actually blocked the word. A wrong or stale label therefore counts as coverage.

FailureRuleAttributor.WordLevelFailureRuleIds is documented as upgrading those labels to verified, but it has no callers.

This is a proposal. #480 is adding two weaker steps:

  • the gate fails when a label is not a rule id in the fixture's grammar;
  • the docs say that blocked_by is asserted, not verified.

This issue tracks the stronger step: verify each label against the parse trace.

Who is affected: maintainers of the HermitCrab conformance suite (#480) and its PanGloss port.

Done when:

  • the gate requires the trace to attribute each word's failure to the rule named in blocked_by;
  • the gate keeps an explicit, reviewed exemption list for words whose failure cannot be attributed from the trace (the attributor's doc names zimed);
  • a mutated label fails the gate.
Code, on #480 at b49433d7
  • CoverageReport.cs:113-114 adds every word.BlockedBy id to the exercised set, with no check.
  • FailureRuleAttributor.cs:66 defines WordLevelFailureRuleIds. The only reference outside its own file is a string in OwnedSymbolKeyTests.cs:132.
  • At that head, all 108 blocked_by labels name real rule ids in their fixture grammars, so no label is wrong today.
Dominant language
C#
Stars
29
Forks
16
Avg merge
6d 17h
Merged PRs (30d)
11

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from sillsdev/machine

All issues in sillsdev/machine

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.