Should the value of `error` be mutable?
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Domain
- testing
Research direction
Review the perfect-numbers canonical-data.json example and problem-specifications PR 1691, then inspect the immutability CI check and any tests.toml assertions mentioned in the discussion. Decide whether error-only wording changes preserve test meaning; done means the policy is recorded and the relevant validation or assertions are updated consistently.
Written by the indexing model from the issue text.
Description
Example
In perfect-numbers we have this test case:
https://github.com/exercism/problem-specifications/blob/8df6cb2ed8e77aa3ac0c02744cd9ddb83920f5f2/exercises/perfect-numbers/canonical-data.json#L121-L130
And we notice (see https://github.com/exercism/problem-specifications/pull/1691) that we would like to make this change:
{
"uuid": "72445cee-660c-4d75-8506-6c40089dc302",
- "description": "Zero is rejected (not a natural number)",
+ "description": "Zero is rejected (not a positive integer)",
"property": "classify",
"input": {
"number": 0
},
"expected": {
- "error": "Classification is only possible for natural numbers."
+ "error": "Classification is only possible for positive integers."
}
We seem to have decided that description is allowed to mutate when the "meaning is unchanged" (that is, we do not add a reimplements test case for e.g. a simple description-only typo fix).
The question is: should the contents of an error also be considered mutable when the "meaning is unchanged", or should that require a new test case?
Discussion
My initial opinion is something like:
- We said that
descriptionshould be mutable if we don't change the "meaning" of the test. - There are some useful error-message changes that don't change the "meaning" of the test either, and the value of
erroris similar to the "description" of the error. In the limiting case, it seems messy to add a new test case just to fix a typo in anerrorvalue. - Therefore, directly mutating the
errorvalue should similarly be allowed when the "meaning is unchanged".
Drawbacks:
- Is there a track that asserts something about the contents of the error message for this kind of test? Such a track would have to update that assertion, even though their
tests.tomldidn't change. - The CI check for immutability would need to check that
expecteddoesn't change, but make a special case for anerror-only change.
- Dominant language
- Ruby
- Stars
- 362
- Forks
- 566
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 8
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from exercism/problem-specifications
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
exercism/problem-specifications#2686 ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
exercism/problem-specifications#2704 · 21 comments · 8 reactions ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 1-3 hours Newbie friendliness 35/100
exercism/problem-specifications#2153 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
exercism/problem-specifications#2120 · 7 comments ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
exercism/problem-specifications#2108 · 1 comment · 5 reactions ·
Maintainers usually reply within 1 day
All issues in exercism/problem-specifications
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
Bug Ruby Waiting for: Product Owner
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
getsentry/sentry-ruby#3228 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
開発用DockerイメージにPlaywrightのシステム依存を追加するPossibly taken @momocus claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 85/100
sakazuki-org/sakazuki#1453 ·
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100