Nothing fails when a benchmark model fit emits a ConvergenceWarning
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 67/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python, scikit-learn
- Domain
- machine-learning, testing
Research direction
Read tests/test_benchmark.py and faircode/models.py to see how audits and strategies run their model fits. Start by checking the existing benchmark tests and how they handle warnings. Add a slow test covering every audit and strategy with ConvergenceWarning treated as an error; done when a fit stopping at max_iter makes the test fail.
Written by the indexing model from the issue text.
Description
Where: tests/test_benchmark.py, faircode/models.py.
Current gap: #1152 was found by hand: logistic regression stopped at max_iter on two audits and no test noticed. Other strategies' fits (reweighted fits inside in-processing, post-processing calibration) are not exercised under warnings-as-errors.
Suggested approach: Add a slow test that runs every audit and strategy with ConvergenceWarning turned into an error.
Done when: the test fails if any fit stops at max_iter.
- Dominant language
- HTML
- Stars
- 47
- Forks
- 45
- Avg merge
- 3h 17m
- Merged PRs (30d)
- 107
Getting set up
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from yakew7/Fair-Code
-
enhancement help wanted tooling
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
bug testing
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
yakew7/Fair-Code#1169 · 2 comments ·
Maintainers usually reply within 1 day
-
ci community enhancement
Difficulty 4/5 3-5 days Newbie friendliness 52/100
Maintainers usually reply within 1 day
-
community enhancement help wanted
Difficulty 3/5 1-2 days Newbie friendliness 68/100
yakew7/Fair-Code#1199 · 2 comments ·
Maintainers usually reply within 1 day
All issues in yakew7/Fair-Code
Similar issues
-
bug help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 2 days
-
[Bug] Math reward accepts distinct large integers after float normalizationPossibly taken @lll-hhh claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Opening-Science/twin-llm#329 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 79/100
aws-samples/sample-per-model-bedrock#639 ·
Maintainers usually reply within 1 day