`modules containers create`: Wave ignores `basePackages`, because the request puts it at the top level
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
Research direction
Read nf_core/modules/containers.py around lines 698–709, where the Wave payload is assembled, and check the container creation path used by bump-versions as well. Move the base package setting into packages.pixiOpts with the package value described in the issue. Run the relevant container payload tests; done when the generated request nests basePackages correctly and no longer sends it at the top level.
Written by the indexing model from the issue text.
Description
Description
nf-core modules containers create (and bump-versions) sends basePackages at the top level of the Wave request. Wave reads basePackages only from packages.pixiOpts (or packages.condaOpts for the micromamba templates). Wave ignores the top-level field, and it uses the default base packages.
Thus the images do not get gzip from conda, as #4449 intended. At the moment, the images have /usr/bin/gzip but only because the final stage of the conda/pixi:v1 template is ubuntu:24.04.
Current code
nf_core/modules/containers.py L698-L709
payload: dict = {
"packages": {
"type": "CONDA",
"environment": base64.b64encode(conda_bytes).decode(),
},
"containerPlatform": platform,
"freeze": True,
"buildTemplate": "conda/pixi:v1",
"basePackages": "conda-forge::procps conda-forge::gzip",
...
}
Why Wave ignores it
- In the Wave API,
SubmitContainerTokenRequesthas no top-levelbasePackagesfield. Wave ignores fields that it does not know. basePackagesis a field ofPixiOpts,CondaOptsandCranOpts. These are inPackagesSpec(packages.pixiOpts,packages.condaOptsandpackages.cranOpts).- If
packages.pixiOpts.basePackagesis not set, Wave usesPixiOpts.DEFAULT_PACKAGES = "conda-forge::procps-ng". - The
conda-pixi-v1template addspixi add conda-forge::which, then{{base_packages}}.
The Wave API docs (https://docs.seqera.io/wave/api#post-v1alpha2container) show the correct place in the examples. The Pixi example has "basePackages": "conda-forge::procps-ng" in packages.pixiOpts. But the attribute table lists basePackages with the top-level fields and shows no nesting. So the table is easy to read incorrectly.
Evidence
The following was generated by Claude 🤖, but matches what I have observed.
Reproducible evidence
-
The Dockerfiles that Wave records for builds from
containers createhave only the default base packages. For example, seebd-28ac84a07bcbf122_1(annotsv amd64),bd-8ac092906065ea38_1(annotsv arm64) andbd-45841ba023524097_1(vcfanno amd64):RUN pixi init --import /opt/wave/conda.yml \ && pixi add conda-forge::which \ && pixi add conda-forge::procps-ng \ -
The conda lock files in
.conda-lock/have nogzippackage. -
A top-level
basePackageshas no effect on the build hash. I sent dry-run requests with theannotsv/annotsvenvironment:Top-level basePackagestargetImagecachedconda-forge::procps conda-forge::gzipannotsv:3.5.10--28ac84a07bcbf122true not set annotsv:3.5.10--28ac84a07bcbf122true conda-forge::jqannotsv:3.5.10--28ac84a07bcbf122true -
packages.pixiOpts.basePackageschanges the build hash:packages.pixiOpts.basePackagestargetImagecachednot set annotsv:3.5.10--28ac84a07bcbf122true conda-forge::procps-ng(the default)annotsv:3.5.10--28ac84a07bcbf122true conda-forge::procps-ng conda-forge::gzipannotsv:3.5.10--2da6feab118d57f1false
How to reproduce
Run this from the root of nf-core/modules. The requests use dryRun, so Wave does not build or push an image:
B=$(base64 -w0 modules/nf-core/annotsv/annotsv/environment.yml)
for opts in '' ',"pixiOpts":{"basePackages":"conda-forge::procps-ng conda-forge::gzip"}'; do
curl -s -X POST https://wave.seqera.io/v1alpha2/container \
-H 'Content-Type: application/json' \
-d "{\"packages\":{\"type\":\"CONDA\",\"environment\":\"$B\"$opts},
\"containerPlatform\":\"linux/amd64\",\"freeze\":true,
\"buildTemplate\":\"conda/pixi:v1\",
\"basePackages\":\"conda-forge::procps conda-forge::gzip\",
\"format\":\"docker\",\"nameStrategy\":\"imageSuffix\",\"dryRun\":true}"
echo
done
The first request has basePackages only at the top level, and it gives the existing image (28ac84a07bcbf122). The second request also has basePackages in packages.pixiOpts, and it gives a new image hash.
To see the Dockerfile of a build, use:
curl -s https://wave.seqera.io/v1alpha1/builds/<buildId> \
| python3 -c "import sys,json; print(json.load(sys.stdin)['dockerFile'])"
Proposed fix
Move basePackages into packages.pixiOpts:
payload: dict = {
"packages": {
"type": "CONDA",
"environment": base64.b64encode(conda_bytes).decode(),
"pixiOpts": {"basePackages": "conda-forge::procps-ng conda-forge::gzip"},
},
"containerPlatform": platform,
"freeze": True,
"buildTemplate": "conda/pixi:v1",
...
}
- Use
conda-forge::procps-ng, notconda-forge::procps.procps-ngis the Wave default and the package that the images have now. - The fix changes the container hash for every module. Modules that already have Wave containers from
containers createwill need new containers. - Alternatively if we are happy to rely on the base image since they are the same across singularity/docker builds now (
pixibuild) via this API call, we can simply remove the redundantbasePackages.
Related
- #4449 added
basePackages. - seqeralabs/wave#1113
- Dominant language
- Python
- Stars
- 328
- Forks
- 268
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 39
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from nf-core/tools
-
enhancement
Difficulty 1/5 Under an hour Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
Incorrect if-statement logic in nf-core modules test outputPossibly taken A pull request linked to this issue is open or already merged. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
nf-core/tools#4327 · 4 comments ·
Maintainers usually reply within 1 day
-
stub `.gz` linting is too restrictivePossibly taken @handsomesix claimed this 141 days ago. Openbug good-first-issue
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
nf-core/tools#4231 · 2 comments ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
modules.json - inconsistent formatting across functionsPossibly taken @harshitethic claimed this 6 days ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
nf-core/tools#4177 · 2 comments ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
vllm-project/vllm#64845 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
dekobon/big-code-analysis#1729 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
pydantic/pydantic-ai#12091 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100