Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Specify dependency of each test case

Open
#51 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
testing, tooling

Research direction

Start by reviewing the test methods and the existing array_helpers and module stubbing mechanisms. Then investigate pytest's collection hook and how the suite represents top-level functions, array object methods, and operators. Done should include a defined dependency representation, a dependency graph, and dependency-aware test prioritization or skipping.

Written by the indexing model from the issue text.

Description

low priority

The test suite is mostly made up of test methods for each function (or array object method) in the spec. The majority of these tests require other functions to do everything we want, which is problematic when an underlying function does not work—you'll get a lot of spammy errors that actually relate to a single fundamental problem... or even false positives.

So I think it would be a good idea if we declared the dependencies of each test method, i.e. the functions we use and assume has correct behaviour. We could use these declarations to create a dependency graph, which could be hooked with pytest to prioritise zero/low-dependency tests first, and by default skipping tests that use functions we've deemed incorrect. This will benefit:

  1. Array API adoptors, who would much more easily see what they should prioritise developing
  2. Us, who'd be able to see areas where we can try to cut down the functions we depend on

Ideas for declaring dependencies per test method:

  • Decorator which lists function and method names
  • Automatically infer dependencies via looking at function code (either searching the code as string, or using the AST)

General challenges would be:

  • How to get pytests collection hook to do this
  • Supporting array object methods alongside the top-level functions
    • As well as operator symbols

The dependency graph + auto skipping would also allow us to:

  • Remove some mirrored functions in array_helpers
  • Remove the module stubbing mechanism
  • Check that no tests is using the function they're testing for assertions
Dominant language
Python
Stars
74
Forks
54
Avg merge
4h 35m
Merged PRs (30d)
4

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from data-apis/array-api-tests

All issues in data-apis/array-api-tests

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.