Problem with resolution
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
Research direction
Reproduce with the provided python-inspector CLI commands and resolver_api calls using re.txt and the PyHEADTAIL setup.py tarball. Start from the stack trace in python_inspector/api.py and python_inspector/resolution.py around get_requirements_from_python_manifest. Done means invalid requirements do not fail silently, setup.py resolution behavior is explained or fixed, and partial resolution/error handling is covered by tests.
Written by the indexing model from the issue text.
Description
There are a couple issues:
1. this file is not a valid requirements file IMHO and we do not return any error:
I save this in re.txt and ran:
python-inspector --json - -p 3.10 -o linux --version -r re.txt
numpy>=1.0
scipy
xobjects
xpart
xdeps
[tests]
cpymad
PyHEADTAIL
pytest
This was copied by mistake as is from the setup.py of xtrack-0.23.2.tar.gz from https://pypi.org/project/xtrack/#files
2. trying to inspect it yields this error which is not right becuase this setup.py is perfectly parsable:
>>> resolver_api(requirement_files=["re.txt"], python_version="310", operating_system="linux")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/api.py", line 254, in resolve_dependencies
resolution, purls = resolve(
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/api.py", line 313, in resolve
resolved_dependencies, packages = get_resolved_dependencies(
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/api.py", line 351, in get_resolved_dependencies
resolver_results = resolver.resolve(requirements=requirements, max_rounds=max_rounds)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 521, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 402, in resolve
failure_causes = self._attempt_to_pin_criterion(name)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 238, in _attempt_to_pin_criterion
criteria = self._get_updated_criteria(candidate)
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/resolvelib/resolvers.py", line 228, in _get_updated_criteria
for requirement in self._p.get_dependencies(candidate=candidate):
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 600, in get_dependencies
return list(self._iter_dependencies(candidate))
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 591, in _iter_dependencies
for r in self.get_requirements_for_package(purl=purl, candidate=candidate):
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 470, in get_requirements_for_package_from_pypi_simple
yield from get_requirements_from_python_manifest(
File "/tmp/scancode-toolkit/venv/lib/python3.8/site-packages/python_inspector/resolution.py", line 304, in get_requirements_from_python_manifest
raise Exception(
Exception: Unable to collect setup.py dependencies securely: .cache/thirdparty/extracted_sdists/PyHEADTAIL-1.16.0/PyHEADTAIL-1.16.0/setup.py
3. Trying to call the API with the setup.py is not right:
See https://files.pythonhosted.org/packages/7c/dd/80d8ec272179941bf6cc181446127da194eb82b7138549a6939583c4c318/PyHEADTAIL-1.16.0.tar.gz (which should have been in the graph above) returns an imcomplete deps tree (or rather nothing)
>>> resolver_api(setup_py_file=".cache/thirdparty/extracted_sdists/PyHEADTAIL-1.16.0/PyHEADTAIL-1.16.0/setup.py", python_version="310", operating_system="linux")
Resolution(resolution=[], packages=[], files=[{'type': 'file', 'path': '.cache/thirdparty/extracted_sdists/PyHEADTAIL-1.16.0/PyHEADTAIL-1.16.0/setup.py', 'package_data': [{'type': 'pypi', 'namespace': None, 'name': 'PyHEADTAIL', 'version': '1.16.0', 'qualifiers': {}, 'subpath': None, 'primary_language': 'Python', 'description': 'CERN PyHEADTAIL numerical n-body simulation code for simulating macro-particle beam dynamics with collective effects.', 'release_date': None, 'parties': [{'type': 'person', 'role': 'author', 'name': 'Kevin Li', 'email': '[email protected]', 'url': None}, {'type': 'person', 'role': 'maintainer', 'name': 'Adrian Oeftiger', 'email': '[email protected]', 'url': None}], 'keywords': [], 'homepage_url': 'https://github.com/PyCOMPLETE/PyHEADTAIL', 'download_url': None, 'size': None, 'sha1': None, 'md5': None, 'sha256': None, 'sha512': None, 'bug_tracking_url': None, 'code_view_url': None, 'vcs_url': None, 'copyright': None, 'license_expression': None, 'declared_license': {}, 'notice_text': None, 'source_packages': [], 'file_references': [], 'extra_data': {}, 'dependencies': [{'purl': 'pkg:pypi/h5py', 'extracted_requirement': 'h5py', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}, {'purl': 'pkg:pypi/numpy', 'extracted_requirement': 'numpy', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}, {'purl': 'pkg:pypi/scipy', 'extracted_requirement': 'scipy', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}, {'purl': 'pkg:pypi/cython', 'extracted_requirement': 'cython', 'scope': 'setup', 'is_runtime': True, 'is_optional': False, 'is_resolved': False, 'resolved_package': {}, 'extra_data': {}}], 'repository_homepage_url': 'https://pypi.org/project/PyHEADTAIL', 'repository_download_url': 'https://pypi.org/packages/source/P/PyHEADTAIL/PyHEADTAIL-1.16.0.tar.gz', 'api_data_url': 'https://pypi.org/pypi/PyHEADTAIL/1.16.0/json', 'datasource_id': 'pypi_setup_py', 'purl': 'pkg:pypi/[email protected]'}]}])
4. Scanning that seame setup.py with SCTK works without a glitch with these deps
dependencies:
- purl: pkg:pypi/h5py
extracted_requirement: h5py
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
- purl: pkg:pypi/numpy
extracted_requirement: numpy
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
- purl: pkg:pypi/scipy
extracted_requirement: scipy
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
- purl: pkg:pypi/cython
extracted_requirement: cython
scope: setup
is_runtime: yes
is_optional: no
is_resolved: no
resolved_package: {}
extra_data: {}
5. running this does not have the extra requires:
python-inspector --spec "xtrack==0.23.3" --json - -p 3.10 -o linux
"resolved_dependencies_graph": [
{
"package": "pkg:pypi/[email protected]",
"dependencies": [
"pkg:pypi/[email protected]"
]
},
{
"package": "pkg:pypi/[email protected]",
"dependencies": []
},
{
"package": "pkg:pypi/[email protected]",
"dependencies": []
},
{
"package": "pkg:pypi/[email protected]",
"dependencies": []
},
{
"package": "pkg:pypi/[email protected]",
"dependencies": [
"pkg:pypi/[email protected]"
]
},
{
"package": "pkg:pypi/[email protected]",
"dependencies": [
"pkg:pypi/[email protected]"
]
},
{
"package": "pkg:pypi/[email protected]",
"dependencies": [
"pkg:pypi/[email protected]",
"pkg:pypi/[email protected]"
]
},
{
"package": "pkg:pypi/[email protected]",
"dependencies": [
"pkg:pypi/[email protected]",
"pkg:pypi/[email protected]",
"pkg:pypi/[email protected]"
]
},
{
"package": "pkg:pypi/[email protected]",
"dependencies": [
"pkg:pypi/[email protected]",
"pkg:pypi/[email protected]",
"pkg:pypi/[email protected]",
"pkg:pypi/[email protected]",
"pkg:pypi/[email protected]"
]
}
]
6. we should have a way to return partial resolutions and we should not fail silently
- Dominant language
- Python
- Stars
- 23
- Forks
- 29
- Avg merge
- 1h 7m
- Merged PRs (30d)
- 2
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from aboutcode-org/python-inspector
-
`packages` is empty when using a non-pypi.org simple index (e.g. Sonatype Nexus)Possibly taken @avika269 claimed this 6 days ago. Open
Difficulty 3/5 Half a day Newbie friendliness 62/100
aboutcode-org/python-inspector#268 · 2 comments ·
-
InconsistentCandidate: Pre-Release not allowMay be free again @Anurag-S1ngh claimed this 25 days ago, and no pull request is open. Openbug design nice-to-have
Difficulty 4/5 3-5 days Newbie friendliness 35/100
aboutcode-org/python-inspector#257 · 1 comment ·
-
Python Inspector version 0.15.0 contains the wrong version stringMay be free again @Anurag-S1ngh claimed this 18 days ago, and no pull request is open. Open
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
aboutcode-org/python-inspector#253 · 2 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 52/100
aboutcode-org/python-inspector#246 · 2 comments · 2 reactions ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
aboutcode-org/python-inspector#242 · 1 comment ·
All issues in aboutcode-org/python-inspector
Similar issues
-
opensubtitlescom: provider rejects pt-PT even though its own converter maps itPossibly taken @paulo-0308 claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Diaoul/subliminal#1405 ·
-
Internal ErrorOpenbug
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
frappe/frappe_docker#2071 ·
Maintainers usually reply within 1 day
-
bug python
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
traceloop/openllmetry#4717 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
The-Pocket/PocketFlow#161 ·
-
comp/desktop P2 type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NousResearch/hermes-agent#157702 ·
Maintainers usually reply within 1 day