Same package versions published as different .deb files in the CUDA repository and nvidia.github.io, so apt reinstalls them in a loop

Open
  • Is something wrong?
#2,137 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
47/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
debian, go
Domain
devops

Research direction

Start by comparing the package publishing paths for the toolkit repository and the CUDA repository; the issue names no source files or tests. Reproduce on Debian 13 or 14 (amd64) with both repositories enabled and the stated pin priorities, then compare package checksums and apt-get -s full-upgrade before and after installation. Done means the same package version no longer triggers repeated reinstall or downgrade plans.

Written by the indexing model from the issue text.

Description

The container toolkit packages are published in two NVIDIA apt repositories: the toolkit's own (https://nvidia.github.io/libnvidia-container/stable/deb/amd64) and the CUDA network repository (https://developer.download.nvidia.com/compute/cuda/repos/debian13/x86_64). For the same version, the two .deb files differ: the CUDA repository's copy carries an extra ar member, _gpgbuilder (an embedded signature), so its size and checksums differ, while control.tar.xz and data.tar.xz are byte-identical.

Checked on 2026-10-11 for libnvidia-container1, libnvidia-container-tools, nvidia-container-toolkit-base and nvidia-container-toolkit, versions 1.20.1-1 and 1.20.0-1: all eight pairs have different SHA256 sums. For example, libnvidia-container1 1.20.1-1 is 1236052 bytes with SHA256 9d803cd9c898… from nvidia.github.io, and 1237398 bytes with SHA256 6c968fc75aa5… from the CUDA repository.

What happens: apt treats the two files as two builds of the same version. The installed package carries no origin in dpkg's status, so apt associates it with one of them, here always the CUDA repository's, even right after installing the nvidia.github.io build. When nvidia.github.io has the higher pin, apt plans to install that build again at every upgrade, reported as a downgrade, and installing it does not end the loop: apt full-upgrade keeps showing "4 downgraded" on a machine that uses both CUDA and the container toolkit.

To reproduce (Debian 13 or 14, amd64): enable both repositories, install nvidia-container-toolkit, pin nvidia.github.io above developer.download.nvidia.com (for example 1003 and 1002), then run apt-get -s full-upgrade, install, and run it again.

Expected: one version, one file. Possible fixes on your side: publish the same .deb (same checksum) in both repositories, keep the toolkit in only one of them, or give the CUDA repository's builds a distinct version (for example a +cuda suffix) so apt can tell them apart.

A workaround that works here: package-specific pins giving both origins the same priority below 1000, so the newer version wins from either repository and an identical version never flips.

Package: libnvidia-container1 libnvidia-container-tools nvidia-container-toolkit nvidia-container-toolkit-base
Pin: origin nvidia.github.io
Pin-Priority: 990

Package: libnvidia-container1 libnvidia-container-tools nvidia-container-toolkit nvidia-container-toolkit-base
Pin: origin developer.download.nvidia.com
Pin-Priority: 990

Environment: Debian testing (forky), apt 3.3.3, toolkit 1.20.1-1.

This issue was written with AI assistance, which I directed and checked.

Daniel Campos Ramos

Dominant language
Go
Stars
4.6k
Forks
609
Avg merge
3d 8h
Merged PRs (30d)
36

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/nvidia-container-toolkit

All issues in NVIDIA/nvidia-container-toolkit

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.