Generate target_compatible_with for whl_library_targets
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 52/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 冷清
- 技术栈
- python
- 领域
- build-system
调研方向
从 python/private/pypi/whl_library_targets.bzl 开始,检查 issue 中提到的 parse_whl_name 工具,并追踪 wheel 文件名如何传递到生成的 py_library 目标。完成的标准是:特定于平台的 wheel 标签会生成相应的 target_compatible_with 约束,而 pure-Python wheel 保持不受约束;验证生成的 Bazel 目标及相关的 build 行为。
由索引模型根据 Issue 内容生成。
描述
🚀 feature request
Relevant Rules
whl_library_targets macro in python/private/pypi/whl_library_targets.bzl
Description
When rules_python generates BUILD targets for platform-specific wheels (e.g., torch-2.8.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl), the resulting py_library does not set target_compatible_with. This means Bazel cannot distinguish platform-specific wheels from pure-Python ones at analysis time.
We maintain a repo that targets both x86_64 and aarch64. Our pyproject.toml uses PEP 508 environment markers to restrict certain dependencies to x86_64:
dependencies = [
"torch==2.8.0; platform_machine == 'x86_64'",
"opencv-python==4.11.0.86; platform_machine == 'x86_64'",
"tensorboard==2.20.0; platform_machine == 'x86_64'",
# ... many more x86_64-only deps
]
We generate separate lock files per platform via uv pip compile. The pip.parse extension correctly fetches platform-specific wheels. However, the generated py_library targets for these wheels have no target_compatible_with constraint.
This causes two issues:
-
bazel build //...fails on aarch64: Bazel attempts to build targets that transitively depend on x86_64-only wheels, which fail because the wheel files don't exist for aarch64. Withtarget_compatible_with, Bazel would instead skip incompatible targets (printingSKIPPEDin the build output). -
Manual
target_compatible_withannotations required everywhere: As a workaround, we must addtarget_compatible_with = ["@platforms//cpu:x86_64"]to everypy_testandpy_binarythat transitively depends on an x86_64-only wheel. This is error-prone and tedious — our repo currently has dozens of such annotations scattered across BUILD files.
Describe the solution you'd like
The whl_library_targets macro should automatically infer target_compatible_with from the wheel filename's platform tag and set it on the generated py_library.
Wheel filenames follow a well-defined format (PEP 427): {distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl. The platform tag encodes the target architecture:
| Platform tag suffix | Constraint |
|---|---|
_x86_64, _amd64 |
@platforms//cpu:x86_64 |
_aarch64, _arm64 |
@platforms//cpu:aarch64 |
_i686 |
@platforms//cpu:x86_32 |
_ppc64le |
@platforms//cpu:ppc |
_s390x |
@platforms//cpu:s390x |
any (pure-Python) |
(no constraint) |
A possible implementation in whl_library_targets.bzl:
load(":parse_whl_name.bzl", "parse_whl_name")
_PLATFORM_TAG_CPU_MAP = {
"x86_64": "@platforms//cpu:x86_64",
"amd64": "@platforms//cpu:x86_64",
"aarch64": "@platforms//cpu:aarch64",
"arm64": "@platforms//cpu:aarch64",
"i686": "@platforms//cpu:x86_32",
"ppc64le": "@platforms//cpu:ppc",
"s390x": "@platforms//cpu:s390x",
}
def _target_compatible_with_from_whl_name(whl_filename):
if not whl_filename or not whl_filename.endswith(".whl"):
return []
parsed = parse_whl_name(whl_filename)
if parsed.platform_tag == "any":
return []
for suffix, constraint in _PLATFORM_TAG_CPU_MAP.items():
if parsed.platform_tag.endswith("_" + suffix):
return [constraint]
return []
Then in the whl_library_targets function, pass it to py_library:
rules.py_library(
...
target_compatible_with = _target_compatible_with_from_whl_name(name),
...
)
This leverages the existing parse_whl_name utility already available in rules_python.
Describe alternatives you've considered
-
Manual
target_compatible_withon downstream targets: This is what we currently do — everypy_test/py_binarythat transitively depends on an x86_64-only wheel gets a manualtarget_compatible_with = ["@platforms//cpu:x86_64"]. This is tedious, error-prone, and doesn't scale. Each new test file requires the developer to figure out whether any transitive dependency is platform-specific. -
Patching
rules_pythonlocally viasingle_version_override: We currently apply a local patch towhl_library_targets.bzl(viasingle_version_overrideinMODULE.bazel) that implements the solution described above. This works but is a maintenance burden — we have to keep the patch in sync with upstreamrules_pythonupdates. -
Using
pip.parsewhl_modificationsto post-hoc addtarget_compatible_with: This would require listing every platform-specific wheel individually and is even more error-prone than option 1.
The information to infer target_compatible_with is already embedded in the wheel filename. Having rules_python set it automatically would eliminate all three workarounds and make cross-platform Bazel builds work correctly out of the box.
- 主要语言
- Starlark
- 星标
- 690
- 派生
- 722
- 平均合并
- 1 天 55 分钟
- 30 天内合并 PR
- 38
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
bazel-contrib/rules_python 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 70/100
bazel-contrib/rules_python#4179 ·
-
难度 2/5 1-3 小时 新手友好度 78/100
bazel-contrib/rules_python#4164 · 1 条评论 ·
-
难度 2/5 1-3 小时 新手友好度 72/100
bazel-contrib/rules_python#3821 ·
-
难度 3/5 1-2 天 新手友好度 45/100
bazel-contrib/rules_python#4181 ·
-
Release 2.4.0 未关闭type: release
难度 4/5 3-5 天 新手友好度 25/100
bazel-contrib/rules_python#4175 · 3 条评论 ·
查看 bazel-contrib/rules_python 的全部 Issue
相似的 Issue
-
难度 1/5 1 小时以内 新手友好度 90/100
-
bug
难度 2/5 1-3 小时 新手友好度 75/100
typelevel/sbt-typelevel#929 ·
-
难度 2/5 1-3 小时 新手友好度 70/100
openSUSE/python-rpm-macros#219 ·
-
难度 2/5 1-3 小时 新手友好度 75/100
-
难度 1/5 1 小时以内 新手友好度 90/100
Qiskit/mcp-servers#221 ·