evidencectl test: fixture category coverage depends on an undocumented case-id convention and does not name the missing category
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 70/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- rust
- Domain
- documentation, testing
Research direction
The check is in crates/registry-evidence/src/bundle.rs in the observe() function. Examine the logic that validates fixture case IDs against the expected categories. Update the error message to list which categories are missing. Also, find where fixture authoring is documented (likely in docs or a skills guide) and add the naming convention there. Run the existing tests for evidencectl test to verify the new error message is helpful.
Written by the indexing model from the issue text.
Description
What happens
evidencectl test refuses with fixture category coverage is incomplete unless the fixture file contains case ids that follow a naming convention (crates/registry-evidence/src/bundle.rs, observe()):
- exactly
positive,no-match,source-failure,anti-reconstruction; - ids starting with
negative,boundary,missing,ambiguous; - a case with
declaredUnresolved: truecovers both no-match and ambiguous.
Why this is a problem
- The convention is not documented in the Evidence authoring docs or skills (only anti-reconstruction is mentioned).
- The error does not name the missing category. An author who named a case
source-timeoutinstead ofsource-failurespent many iterations and concluded the check always fails.
Ask
Name the missing categories in the error, and document the id convention where fixtures are taught. Consider an explicit category: field instead of id prefixes.
- Dominant language
- Rust
- Stars
- 2
- Forks
- 0
- Avg merge
- 4h 26m
- Merged PRs (30d)
- 152
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from registrystack/registry-stack
-
bug github_actions
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
registrystack/registry-stack#1393 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
registrystack/registry-stack#1383 ·
-
area:breg enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
registrystack/registry-stack#1380 ·
-
area:breg enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
registrystack/registry-stack#1379 ·
-
area:casework documentation
Difficulty 1/5 Under an hour Newbie friendliness 90/100
registrystack/registry-stack#1369 ·
All issues in registrystack/registry-stack
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
TheLarkInn/aipm#2413 ·
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 90/100
alexgorbatchev/simple-ptt#15 ·
-
tooling
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
todo:ticket
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
taikoxyz/taiko-mono#22168 · 1 comment ·