pip extension facts structure prevents correct git merge of MODULE.bazel.lock
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python
- Domain
- build-system
Research direction
Start in python/extensions:pip.bzl and inspect how pip facts are written into MODULE.bazel.lock, then read scripts/bazel-lockfile-merge.jq to confirm the shallow merge behavior. Reproduce concurrent dependency additions and verify that the merged lockfile retains both packages and remains up to date.
Written by the indexing model from the issue text.
Description
Summary
The pip extension stores its facts with the index URL as a top-level key and package names one level deeper:
{
"@@rules_python+//python/extensions:pip.bzl%pip": {
"dist_hashes": {
"https://pypi.org/simple/": {
"package-a": { "wheel_path": "sha256:..." },
"package-b": { "wheel_path": "sha256:..." }
}
},
"index_urls": {
"https://pypi.org/simple/": {
"package-a": "package-a"
}
}
}
}
Bazel ships a git merge driver for MODULE.bazel.lock (scripts/bazel-lockfile-merge.jq) that merges the facts section using shallow_merge, which applies last-wins semantics at the top level only (jq add).
Because dist_hashes and index_urls are single top-level keys shared by all packages, when two branches each add a different package, one branch's entire dist_hashes map overwrites the other's. The merged lockfile is missing the packages from one branch, causing:
MODULE.bazel.lock is no longer up-to-date because the extension
'@@rules_python+//python/extensions:pip.bzl%pip' has changed its facts
Root cause
This is a mismatch between the facts structure and the merge driver's semantics. I filed https://github.com/bazelbuild/bazel/issues/31139 requesting a deep-merge for facts, but the Bazel team considers shallow_merge intentional — deep-merging arbitrary extension facts could be wrong for other extensions — and redirected to rules_python.
Suggested fix
If the facts structure used package names (or index_url + package name) as top-level keys, shallow_merge would correctly preserve entries from both branches:
{
"@@rules_python+//python/extensions:pip.bzl%pip": {
"https://pypi.org/simple/ package-a": { "wheel_path": "sha256:..." },
"https://pypi.org/simple/ package-b": { "wheel_path": "sha256:..." }
}
}
Any flattening that puts per-package data under distinct top-level keys would solve the problem.
Impact
This affects any team that uses a CI setup where master is merged into PR branches before running tests (common in Jenkins-based pipelines), and where two concurrent PRs each add a Python dependency.
- Dominant language
- Starlark
- Stars
- 690
- Forks
- 722
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 50
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from bazel-contrib/rules_python
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
bazel-contrib/rules_python#4164 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
bazel-contrib/rules_python#3821 ·
-
Release 2.4.0 Opentype: release
Difficulty 4/5 3-5 days Newbie friendliness 25/100
bazel-contrib/rules_python#4175 · 3 comments ·
-
Release 2.3.4 Opentype: release
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
bazel-contrib/rules_python#4173 · 2 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 70/100
bazel-contrib/rules_python#4171 ·
All issues in bazel-contrib/rules_python
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
area:engineering priority:p2 type:bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
jejjohnson/pyrox#234 ·
-
good first issue
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
good first issue
Difficulty 1/5 Under an hour Newbie friendliness 85/100
ros2/ros2_tracing#266 · 1 comment ·