Platform-specific packages missing from the unified `@pypi` hub

Open
#4,171 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
70/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python

Research direction

Start at the pip.parse entry point and reproduce the issue with MODULE.bazel, BUILD.bazel, requirements_linux.txt, and requirements_windows.txt. Compare the concrete @pypi_b//six alias with @pypi//six, then add regression coverage showing that the unified hub includes platform-specific packages and the alias resolves on Linux.

Written by the indexing model from the issue text.

Description

🐞 bug report

(This was diagnosed with help from Codex.)

Affected Rule

pip.parse and the automatically generated unified @pypi hub.

Is this a regression?

Not sure.

Description

I'm seeing an issue where packages that appear only in some platform-specific requirements files are omitted from the unified @pypi hub.

I have an example where I expect @pypi//six to resolve on Linux when the selected concrete hub provides six for Linux. Instead, the unified hub omits the package entirely, even though the concrete hub provides the alias.

The unified hub collects packages from exposed_packages, which filters out platform-specific packages. I think it should collect packages from each concrete hub's complete wheel map and let that hub's aliases handle platform selection.

🔬 Minimal Reproduction

With rules_python and a Python 3.11 toolchain configured, add:

# MODULE.bazel
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")

pip.parse(
    hub_name = "pypi_b",
    python_version = "3.11",
    requirements_by_platform = {
        "//:requirements_linux.txt": "linux_*",
        "//:requirements_windows.txt": "windows_*",
    },
    target_platforms = [
        "linux_x86_64",
        "windows_x86_64",
    ],
)

pip.default(default_hub = "pypi_b")

use_repo(pip, "pypi_b", "pypi")
# BUILD.bazel
exports_files([
    "requirements_linux.txt",
    "requirements_windows.txt",
])
# requirements_linux.txt
colorama==0.4.5
six==1.16.0
# requirements_windows.txt
colorama==0.4.5

On Linux x86_64, compare:

bazel build @pypi_b//six
bazel build @pypi//six

The concrete alias resolves, but the unified alias is absent.

🔥 Exception or Error

Bazel cannot resolve @pypi//six because the unified repository does not contain the six package.

🌍 Your Environment

Operating system: linux_x86_64.

Bazel version used to validate the fix: 9.1.1.

rules_python @ 9cc620ebf92d9afaa1c0535968d82d5a7d7e4522.

Anything else relevant?

Using the concrete hub label avoids the missing unified alias. I'd be happy to contribute the fix and regression coverage.

Dominant language
Starlark
Stars
690
Forks
722
Avg merge
1d 2h
Merged PRs (30d)
50

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from bazel-contrib/rules_python

All issues in bazel-contrib/rules_python

Similar issues

More Build System issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.