Add partial filtering
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 18/100
Research direction
The issue names no specific files or tests. It says the current filtering discards a whole test when any element is selected, and that analysis should return metric dictionaries callable outside pytest. Start by reading the existing filter and analysis code in the repository, then decide the design with maintainers before writing code. Done would mean a written design agreed in the thread, since the issue itself does not define a concrete change.
Written by the indexing model from the issue text.
Description
Summary
Filtering currently works in a basic form, where if any element in a test is selected, the whole test is discarded.
In many cases, it would be much more preferable to exclude specific data points from the metric calculated instead. This requires a few additional changes:
- Analysis most likely should be rewritten in a form where the metric dictionary can be returned as a function callable outside of
pytest(so so reducing dependencies on filters) - Applications will need custom filter functions, which will require processing of data to pass to the metric getter function. For example, this could be a masked list of errors from data taken from the app's scatter plot.
This could become quite expensive to run.
Example
No response
Alternatives
No response
Additional context
No response
- Dominant language
- Python
- Stars
- 69
- Forks
- 68
- Avg merge
- 3d 23h
- Merged PRs (30d)
- 23
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ddmms/ml-peg
-
Sort modelsOpenenhancement good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
“My Pareto front”Openenhancement
Difficulty 5/5 Over a week Newbie friendliness 15/100
ddmms/ml-peg#948 · 1 comment ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
aws-samples/sample-per-model-bedrock#243 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
FuRongJun-1999/dsh-memory#404 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
bug frontend
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
PennyLaneAI/catalyst#3337 ·
Maintainers usually reply within 1 day
-
Workflow.schedule stops when a cron deadline passes before sleepPossibly taken @HuaTNA claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day