The benchmark does not record how long each fit takes
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 82/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python
- Domain
- testing-qa
Research direction
Read faircode/benchmark.py at run_audit and collect_provenance, then inspect the progress_callback and provenance-writing paths in faircode/cli.py. Use a stub clock in the relevant test to check per-combination timing values. Done means each run records those seconds in PROVENANCE.json, diff_results.py ignores the added timings, and the existing schema check passes.
Written by the indexing model from the issue text.
Description
Where: faircode/benchmark.py (run_audit, collect_provenance), faircode/cli.py.
Current gap: The only timing in a benchmark run is what the user sees on the clock: a full run takes 5 to 15 minutes and the in-processing fit on Healthcare Readmission alone about 98 s (closed as #1238), but nothing records which (audit, strategy, model) combination took how long, so a slowdown after a dependency bump (see the rerun-benchmark label) cannot be located.
Suggested approach: Time each combination with time.perf_counter in run_audit and write the seconds into the existing progress_callback call and a timings list in the new run's PROVENANCE.json entry (additive, so the existing schema check keeps passing).
Done when: a run writes per-combination seconds into PROVENANCE.json, diff_results.py ignores them, and a test with a stub clock checks the values.
- Dominant language
- HTML
- Stars
- 47
- Forks
- 45
- Avg merge
- 2h 49m
- Merged PRs (30d)
- 69
Getting set up
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from yakew7/Fair-Code
-
enhancement tooling
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
community enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
Maintainers usually reply within 1 day
-
The weekly external-links issue counts the 403 links with no archive copy but does not list themOpenci enhancement tooling
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
Maintainers usually reply within 1 day
-
enhancement good first issue testing
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
check_contributors.py --stub does not say when a pull request's commit is authored by someone elseOpencommunity enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
All issues in yakew7/Fair-Code
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Maintainers usually reply within 1 day
-
good first issue help wanted
Difficulty 1/5 Under an hour Newbie friendliness 87/100
Maintainers usually reply within 1 day
-
audit-readme-drift
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
jolars/panache#627 · 1 comment ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
swc-project/swc-next-issues#490 ·