Skipped flaky tests
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
Research direction
Start by locating and running the named tests: test_reshape, test_std, test_var, test_remainder, and test_count_nonzero. Inspect why the tests are skipped or flaky across implementations, especially JAX, while leaving timeout health checks skipped and addressing filtering health checks through the strategy. Done means the listed tests run reliably without inappropriate skips and still verify the intended behavior.
Written by the indexing model from the issue text.
Description
Several tests are completely skipped right now because they are "flaky".
test_reshapetest_stdtest_vartest_remainder
This is a pretty high priority issue because these functions are effectively completely untested, even though they appear to be tested.
Tests should be written in such a way that they aren't flaky, for instance, by using high numerical tolerances (or if necessary, avoiding values testing entirely).
Note that health checks for timeouts should just be skipped, and health checks for filtering too much should be fixed by fixing the strategy.
EDIT:
test_count_nonzerois not skipped but is flaky on JAX. Some discussion is at https://github.com/data-apis/array-api-tests/pull/347#issuecomment-2695654862
- Dominant language
- Python
- Stars
- 74
- Forks
- 54
- Avg merge
- 4h 35m
- Merged PRs (30d)
- 4
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from data-apis/array-api-tests
-
Difficulty 2/5 1-3 hours Newbie friendliness 87/100
data-apis/array-api-tests#465 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
data-apis/array-api-tests#460 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
data-apis/array-api-tests#455 · 2 comments ·
-
tracking
Difficulty 5/5 Over a week Newbie friendliness 25/100
data-apis/array-api-tests#381 · 9 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 38/100
data-apis/array-api-tests#379 · 10 comments ·
All issues in data-apis/array-api-tests
Similar issues
-
bug confirmed issue
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
open-webui/open-webui#30750 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
OpenwaterHealth/openmotion-bloodflow-app#604 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
good first issue
Difficulty 1/5 Under an hour Newbie friendliness 90/100