Trouble running tests: Failed to find precompiled modules

Open Beginner friendly
#7,909 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
74/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
testing-qa

Research direction

Start with tests/conftest.py at lines 20-21 and inspect how the source directory is inserted into sys.path. Install DeepSpeed with the DS_BUILD options, then run pytest --forked unit/ from tests; done means the tests load the installed package and can find its precompiled ops.

Written by the indexing model from the issue text.

Description

I want to test my installation like this:

  • DS_BUILD_<...> pip install <--options...> .
  • cd tests
  • pytest --forked unit/

However this doesn't work because deepspeed inserts the source into the syspath: https://github.com/deepspeedai/DeepSpeed/blob/5f7b687018bd1e0340c661859820fd97aa80a616/tests/conftest.py#L20-L21

Consequently it fails to find the installed ops.

Given the recent discovery of issues with (pre)compiled ops I think this should be changed. If someone really wants to test the source without precompiled ops they can modify PYTHONPATH or not install deepspeed and run from the source directory as pytest tests/unit/

Dominant language
Python
Stars
43.1k
Forks
5k
Avg merge
4d 19h
Merged PRs (30d)
116

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from deepspeedai/DeepSpeed

All issues in deepspeedai/DeepSpeed

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.