Parallel `bundle install` races on a git cache shared by multiple gems from one repo

Aperta

#9,946 aperta il 13/10/2026 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@Shaisolaris ci sta già lavorando.

Dal 14/10/2026.

#9951 di @Shaisolaris — aperta

Valutazione

Difficoltà: 4/5
Tempo stimato: 3-5 giorni
Idoneità per principianti: 58/100

Tipo di issue: Bug
Chiarezza: Specificata chiaramente
Stato di attività: Attiva
Stack tecnologico: git, ruby
Ambito: build-system, tooling

Direzione di ricerca

Start in lib/bundler/source/git/git_proxy.rb at the fetch and retry logic linked in the issue, then inspect the parallel git operations added in #9323 and the related fixes in #9186/#9196. Use the supplied Docker reproducer with --jobs 2 and compare with --jobs 1; done when parallel installs from one repository at different revisions reliably install all five gems without the shared-cache fetch failure.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Bundler

Describe the problem as clearly as you can

When several gems are sourced from the same git repository but locked to different revisions (e.g. a monorepo where each gem has its own tag), they all share one cache directory (cache/bundler/git/<repo>-<hash>). Since Bundler 4.0.7 started running git operations in parallel (#9323), bundle install --jobs N (N >= 2) intermittently fails with exit code 11 (Bundler::GitError), because several workers run git fetch against that shared directory at the same time.

From the logs, the failure goes like this:

One worker creates or updates the shallow cache for its revision.
Another worker sees that the cache does not contain its revision and runs git fetch --unshallow on the same directory.
The two collide: fatal: shallow file has changed since we read it or fatal: Unable to create '.../shallow.lock': File exists.
Retrying does not help. Once another worker has unshallowed the cache, every retry of the same command fails with fatal: --unshallow on a complete repository does not make sense. The decision to pass --unshallow appears to be made once in GitProxy#checkout, before the retry loop:
https://github.com/ruby/rubygems/blob/f204bf4d6cb1551d56ae881caefe0a445f5d6c9a/lib/bundler/source/git/git_proxy.rb#L126-L136
After the retries are exhausted, Bundler prints Using cached git data because of network errors, but the cache has no refs/<sha>-sha for that revision. The fetch in copy_to then fails with fatal: couldn't find remote ref refs/<sha>-sha, and the install aborts.

I first hit this on GitHub Actions (ubuntu-24.04, 2 vCPUs, x86_64). ruby/setup-ruby with bundler-cache: true runs bundle install --jobs 2 there. The setup was Bundler 4.0.14 and a private monorepo with four gems locked to four different tags. That run was a cache miss (nothing was restored into vendor/bundle), so it started from an empty git cache:

[command]/opt/hostedtoolcache/Ruby/4.0.5/x64/bin/bundle install --jobs 2
Fetching https://github.com/<org>/<monorepo>.git
Fetching https://github.com/<org>/<monorepo>.git
Retrying `git fetch --force --quiet --no-tags --unshallow -- ***github.com/<org>/<monorepo>.git 8aaf04b5...:refs/8aaf04b5...-sha` at .../cache/bundler/git/<monorepo>-fa8eaece... due to error (2/4): ...
fatal: shallow file has changed since we read it
Retrying `git fetch ... --unshallow ...` ... due to error (3/4): ...
fatal: --unshallow on a complete repository does not make sense
Retrying `git fetch ... --unshallow ...` ... due to error (4/4): ...
fatal: --unshallow on a complete repository does not make sense
Using cached git data because of network errors:
...
Git error: command `git fetch --force --quiet .../cache/bundler/git/<monorepo>-fa8eaece... --depth 1 refs/8aaf04b5...-sha` in directory .../bundler/gems/<monorepo>-8aaf04b5d872 has failed.
fatal: couldn't find remote ref refs/8aaf04b5...-sha
##[error]The process '/opt/hostedtoolcache/Ruby/4.0.5/x64/bin/bundle' failed with exit code 11

Starting from an empty cache, I could not reproduce this reliably with the small public repo below. In those runs the workers mostly collided earlier, at git clone (fatal: destination path '...' already exists and is not an empty directory.). They used up their retries and fell back to cached git data, and a later fetch then succeeded, so the install completed (20/20 attempts, Bundler 4.0.22, --jobs 4). To make the reproducer consistently exercise the --unshallow path that can fail, the script below first leaves behind a shallow cache that contains only one of the gems.

Results of the script (10 attempts each, --jobs 2, RubyGems 4.0.22, Docker on a 2-vCPU aarch64 VM):

Bundler	failed attempts
4.0.6	0/10
4.0.7	6/10
4.0.22 (latest)	6/10

The failure rate varies between runs (another 10-attempt run with 4.0.22 failed 3/10), but every run with 4.0.7 or later that I tried had failures.

With --jobs 1 it never fails (0/10, and no retries at all).

The only workaround I found is serializing with --jobs 1. With ruby/setup-ruby, this required disabling bundler-cache: true and running the cache and install steps separately. ruby/setup-ruby passes --jobs N explicitly, and that overrides BUNDLE_JOBS=1.

Possibly related:

#9323 (Run git operations in parallel (take 2), originally #9100), released in 4.0.7, which matches the version where this starts failing.
#9186 / #9196: the same "multiple gems from the same repo" race, fixed for bundle pristine only.

Did you try upgrading rubygems & bundler?

Yes. Reproduced with Bundler 4.0.22 / RubyGems 4.0.22, the latest stable versions at the time of writing.

Post steps to reproduce the problem

The test repository https://github.com/RyumaRyamaNgo/bundler-git-race-gems contains five minimal gems (race_gem_a ... race_gem_e). Each one was added in its own commit and tagged on that commit, so every tag is a different revision.

docker build -t bundler-git-race .
docker run --rm bundler-git-race 2 10   # --jobs 2, 10 attempts

# compare with other versions
docker build --build-arg BUNDLER_VERSION=4.0.6 -t bundler-git-race:4.0.6 .
docker run --rm bundler-git-race:4.0.6 2 10

Dockerfile

FROM ruby:4.0.7

# ruby:4.0.7 ships RubyGems and Bundler 4.0.20. Update RubyGems to the
# latest stable version and install Bundler explicitly. BUNDLER_VERSION makes
# `bundle` run exactly the requested version so releases can be compared with
# `--build-arg BUNDLER_VERSION=...`.
ARG RUBYGEMS_VERSION=4.0.22
RUN gem update --system "${RUBYGEMS_VERSION}"

ARG BUNDLER_VERSION=4.0.22
RUN gem install bundler -v "${BUNDLER_VERSION}"
ENV BUNDLER_VERSION=${BUNDLER_VERSION}

# Only to keep the output readable (silences git's "initial branch" hint).
RUN git config --global init.defaultBranch main

WORKDIR /app
COPY repro.sh /repro.sh
RUN chmod +x /repro.sh

ENTRYPOINT ["/repro.sh"]

repro.sh

#!/usr/bin/env bash
#
# Reproduction for: parallel `bundle install` fails when several gems come from
# the same git repository but are locked to different revisions.
#
# Each attempt mimics a CI job that restores a cached git checkout:
#   1. install ONE gem from the repo, leaving a shallow git cache behind
#   2. delete the installed gems, keep the git cache
#   3. `bundle install --jobs N` with a lockfile that pins all five gems
#
# Usage: ./repro.sh [JOBS] [ATTEMPTS]
set -eu

JOBS="${1:-4}"
ATTEMPTS="${2:-10}"
REPO="RyumaRyamaNgo/bundler-git-race-gems"

export BUNDLE_PATH=vendor/bundle

write_gemfile() {
  echo 'source "https://rubygems.org"' > Gemfile
  for g in "$@"; do
    echo "gem \"race_gem_${g}\", github: \"${REPO}\", tag: \"race_gem_${g}-v1.0.0\", glob: \"race_gem_${g}/*.gemspec\"" >> Gemfile
  done
}

# Lockfile for the full Gemfile, generated once.
rm -rf vendor Gemfile.lock .bundle
write_gemfile a b c d e
bundle lock > /dev/null 2>&1
cp Gemfile.lock /tmp/Gemfile.lock.full

echo "bundler $(bundle --version | awk '{print $NF}'), ruby $(ruby -e 'print RUBY_VERSION'), $(git --version), nproc $(nproc), --jobs ${JOBS}"

failed=0
for i in $(seq 1 "${ATTEMPTS}"); do
  rm -rf vendor Gemfile.lock .bundle

  # 1. leave a shallow git cache behind
  write_gemfile a
  bundle install --jobs 1 > /dev/null 2>&1

  # 2. drop installed gems, keep vendor/bundle/ruby/*/cache/bundler/git
  rm -rf vendor/bundle/ruby/*/bundler/gems/*

  # 3. parallel install with all five gems locked
  write_gemfile a b c d e
  cp /tmp/Gemfile.lock.full Gemfile.lock
  if bundle install --jobs "${JOBS}" > "/tmp/attempt-${i}.log" 2>&1; then
    echo "attempt ${i}: ok"
  else
    echo "attempt ${i}: FAILED (exit $?)"
    failed=$((failed + 1))
    last_failure="/tmp/attempt-${i}.log"
  fi
done

echo "${failed}/${ATTEMPTS} attempts failed"
if [ -n "${last_failure:-}" ]; then
  echo "----- output of the last failed attempt -----"
  cat "${last_failure}"
fi

if [ "${SHOW_ENV:-0}" = "1" ]; then
  echo "----- bundle env -----"
  bundle env
fi

if [ "${failed}" -gt 0 ]; then
  exit 1
fi

Which command did you run?

bundle install --jobs 2 (step 3 of the script)

What were you expecting to happen?

The five gems are installed, as with --jobs 1 or Bundler 4.0.6.

What happened instead?

bundler 4.0.22, ruby 4.0.7, git version 2.47.3, nproc 2, --jobs 2
attempt 1: FAILED (exit 11)
attempt 2: FAILED (exit 11)
attempt 3: FAILED (exit 11)
attempt 4: FAILED (exit 11)
attempt 5: FAILED (exit 11)
attempt 6: FAILED (exit 11)
attempt 7: ok
attempt 8: ok
attempt 9: ok
attempt 10: ok
6/10 attempts failed

Output of a failed attempt

Fetching https://github.com/RyumaRyamaNgo/bundler-git-race-gems.git
Fetching https://github.com/RyumaRyamaNgo/bundler-git-race-gems.git

Retrying `git fetch --force --quiet --no-tags --unshallow -- https://github.com/RyumaRyamaNgo/bundler-git-race-gems.git 127ad62a67da1a1b1907f1b5dd6d954530731c62:refs/127ad62a67da1a1b1907f1b5dd6d954530731c62-sha` at /app/vendor/bundle/ruby/4.0.0/cache/bundler/git/bundler-git-race-gems-a2a00039137f15574ead9be4bf2055e8c970008a due to error (2/4): Bundler::Source::Git::GitCommandError Git error: command `git fetch --force --quiet --no-tags --unshallow -- https://github.com/RyumaRyamaNgo/bundler-git-race-gems.git 127ad62a67da1a1b1907f1b5dd6d954530731c62:refs/127ad62a67da1a1b1907f1b5dd6d954530731c62-sha` in directory /app/vendor/bundle/ruby/4.0.0/cache/bundler/git/bundler-git-race-gems-a2a00039137f15574ead9be4bf2055e8c970008a has failed.
fatal: Unable to create '/app/vendor/bundle/ruby/4.0.0/cache/bundler/git/bundler-git-race-gems-a2a00039137f15574ead9be4bf2055e8c970008a/shallow.lock': File exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.

Retrying `git fetch --force --quiet --no-tags --unshallow -- https://github.com/RyumaRyamaNgo/bundler-git-race-gems.git 127ad62a67da1a1b1907f1b5dd6d954530731c62:refs/127ad62a67da1a1b1907f1b5dd6d954530731c62-sha` at /app/vendor/bundle/ruby/4.0.0/cache/bundler/git/bundler-git-race-gems-a2a00039137f15574ead9be4bf2055e8c970008a due to error (3/4): Bundler::Source::Git::GitCommandError Git error: command `git fetch --force --quiet --no-tags --unshallow -- https://github.com/RyumaRyamaNgo/bundler-git-race-gems.git 127ad62a67da1a1b1907f1b5dd6d954530731c62:refs/127ad62a67da1a1b1907f1b5dd6d954530731c62-sha` in directory /app/vendor/bundle/ruby/4.0.0/cache/bundler/git/bundler-git-race-gems-a2a00039137f15574ead9be4bf2055e8c970008a has failed.
fatal: --unshallow on a complete repository does not make sense

Retrying `git fetch --force --quiet --no-tags --unshallow -- https://github.com/RyumaRyamaNgo/bundler-git-race-gems.git 127ad62a67da1a1b1907f1b5dd6d954530731c62:refs/127ad62a67da1a1b1907f1b5dd6d954530731c62-sha` at /app/vendor/bundle/ruby/4.0.0/cache/bundler/git/bundler-git-race-gems-a2a00039137f15574ead9be4bf2055e8c970008a due to error (4/4): Bundler::Source::Git::GitCommandError Git error: command `git fetch --force --quiet --no-tags --unshallow -- https://github.com/RyumaRyamaNgo/bundler-git-race-gems.git 127ad62a67da1a1b1907f1b5dd6d954530731c62:refs/127ad62a67da1a1b1907f1b5dd6d954530731c62-sha` in directory /app/vendor/bundle/ruby/4.0.0/cache/bundler/git/bundler-git-race-gems-a2a00039137f15574ead9be4bf2055e8c970008a has failed.
fatal: --unshallow on a complete repository does not make sense

Using cached git data because of network errors:
Git error: command `git fetch --force --quiet --no-tags --unshallow -- https://github.com/RyumaRyamaNgo/bundler-git-race-gems.git 127ad62a67da1a1b1907f1b5dd6d954530731c62:refs/127ad62a67da1a1b1907f1b5dd6d954530731c62-sha` in directory /app/vendor/bundle/ruby/4.0.0/cache/bundler/git/bundler-git-race-gems-a2a00039137f15574ead9be4bf2055e8c970008a has failed.
fatal: --unshallow on a complete repository does not make sense
Git error: command `git fetch --force --quiet
/app/vendor/bundle/ruby/4.0.0/cache/bundler/git/bundler-git-race-gems-a2a00039137f15574ead9be4bf2055e8c970008a
--depth 1 refs/127ad62a67da1a1b1907f1b5dd6d954530731c62-sha` in directory
/app/vendor/bundle/ruby/4.0.0/bundler/gems/bundler-git-race-gems-127ad62a67da
has failed.
fatal: couldn't find remote ref
refs/127ad62a67da1a1b1907f1b5dd6d954530731c62-sha

If not included with the output of your command, run `bundle env` and paste the output below

bundle env

Environment

Bundler       4.0.22
  Platforms   ruby, aarch64-linux
Ruby          4.0.7p0 (2026-09-15 revision 229531a6cfbf07e3caef30dbac24a2a3f3fed482) [aarch64-linux]
  Full Path   /usr/local/bin/ruby
  Config Dir  /usr/local/etc
RubyGems      4.0.22
  Gem Home    /usr/local/bundle
  Gem Path    /root/.local/share/gem/ruby/4.0.0:/usr/local/lib/ruby/gems/4.0.0:/usr/local/bundle
  User Home   /root
  User Path   /root/.local/share/gem/ruby/4.0.0
  Bin Dir     /usr/local/bundle/bin
Tools         
  Git         2.47.3
  RVM         not installed
  rbenv       not installed
  chruby      not installed

Bundler Build Metadata

Timestamp  2026-09-30
Git SHA    ff2bd50

Bundler settings

app_config
  Set via BUNDLE_APP_CONFIG: "/usr/local/bundle"
path
  Set via BUNDLE_PATH: "vendor/bundle"
silence_root_warning
  Set via BUNDLE_SILENCE_ROOT_WARNING: true

Gemfile

source "https://rubygems.org"
gem "race_gem_a", github: "RyumaRyamaNgo/bundler-git-race-gems", tag: "race_gem_a-v1.0.0", glob: "race_gem_a/*.gemspec"
gem "race_gem_b", github: "RyumaRyamaNgo/bundler-git-race-gems", tag: "race_gem_b-v1.0.0", glob: "race_gem_b/*.gemspec"
gem "race_gem_c", github: "RyumaRyamaNgo/bundler-git-race-gems", tag: "race_gem_c-v1.0.0", glob: "race_gem_c/*.gemspec"
gem "race_gem_d", github: "RyumaRyamaNgo/bundler-git-race-gems", tag: "race_gem_d-v1.0.0", glob: "race_gem_d/*.gemspec"
gem "race_gem_e", github: "RyumaRyamaNgo/bundler-git-race-gems", tag: "race_gem_e-v1.0.0", glob: "race_gem_e/*.gemspec"

Gemfile.lock

Only the first GIT block is shown. race_gem_b ... race_gem_e each have their own GIT block, identical except for revision / tag / glob / gem name (same remote, different revision).

GIT
  remote: https://github.com/RyumaRyamaNgo/bundler-git-race-gems.git
  revision: 98f94a58b67a6067f6fa855ee16f8f1702a0ca5f
  tag: race_gem_a-v1.0.0
  glob: race_gem_a/*.gemspec
  specs:
    race_gem_a (1.0.0)

GEM
  remote: https://rubygems.org/
  specs:

PLATFORMS
  aarch64-linux
  ruby

DEPENDENCIES
  race_gem_a!
  race_gem_b!
  race_gem_c!
  race_gem_d!
  race_gem_e!

CHECKSUMS
  bundler (4.0.22) sha256=d8d5ec84c8555e0af71db63ed7aee4d1a8fb839ec46d84212d61979242a5d75a
  race_gem_a (1.0.0)
  race_gem_b (1.0.0)
  race_gem_c (1.0.0)
  race_gem_d (1.0.0)
  race_gem_e (1.0.0)

BUNDLED WITH
  4.0.22

Lingua principale: Ruby
Stelle: 4k
Fork: 1.9k
Merge medio: 1g 9h
PR unite (30g): 37

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Nessun Dockerfile né file Docker Compose
Ha un modello di pull request
Leggi la guida per i contributori

Come iniziare

Leggi tutta la issue e poi la guida ai contributi del progetto.
Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
Fai un fork del repository e lavora su un branch.
Apri una pull request che faccia riferimento al numero della issue.

Altre issue di ruby/rubygems

Uninstalling a gem fails if bin directory doesn't exist
Aperta
- Qualcosa non va?
RubyGems

Difficoltà 2/5 1-3 ore Idoneità per principianti 73/100

ruby/rubygems#9959 · Aperta il 20/10/2026

I maintainer di solito rispondono entro 1 giorno
Record `override` directives in Gemfile.lock (an `OVERRIDES` section)
Forse già presa @paracycle l’ha presa 1 giorno fa. Aperta
- Qualcosa non va?
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100

ruby/rubygems#9957 · 3 commenti · Aperta il 16/10/2026

I maintainer di solito rispondono entro 1 giorno
Gem.load_plugins loads same plugin multiple times from different Gem.path entries
Aperta
- Qualcosa non va?
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100

ruby/rubygems#9945 · 2 reazioni · Aperta il 12/10/2026

I maintainer di solito rispondono entro 1 giorno
Process lock can let two `bundle install` runs in at once when a third process is waiting
Aperta
- Qualcosa non va?
Bundler

Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100

ruby/rubygems#9944 · Aperta il 12/10/2026

I maintainer di solito rispondono entro 1 giorno
Bundler: Concurrent Bundler processes fail on Windows with `PermissionError` on the compact index cache
Aperta
- Qualcosa non va?
Bundler

Difficoltà 4/5 3-5 giorni Idoneità per principianti 30/100

ruby/rubygems#9939 · Aperta il 9/10/2026

I maintainer di solito rispondono entro 1 giorno

Tutte le issue di ruby/rubygems

Issue simili

【问题】Homebrew 加载 snow-shot cask 时提示 preflight 已弃用
Aperta
- Qualcosa non va?
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100

mg-chao/snow-apps#2096 · Aperta il 21/10/2026

I maintainer di solito rispondono entro 1 giorno
Bundled ca-bundle.crt trusts AAA Certificate Services, which cross-signed the compromised eMudhra intermediates
Aperta
- Qualcosa non va?
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100

aws/aws-sdk-ruby#3460 · Aperta il 21/10/2026

I maintainer di solito rispondono entro 1 giorno
Nicoseiga extractor doesn't grab nicovideo URLs linked in commentary
Aperta
- Qualcosa non va?
Difficoltà 2/5 1-3 ore Idoneità per principianti 67/100

danbooru/danbooru#6720 · Aperta il 20/10/2026

I maintainer di solito rispondono entro 17 giorni
State Enable access bits are named incorrectly
Aperta
- Qualcosa non va?
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100

riscv/riscv-unified-db#2755 · Aperta il 20/10/2026

I maintainer di solito rispondono entro 1 giorno
Request: update repo_url for project 12291 (murk) after GitHub org rename
Aperta
- Qualcosa non va?
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100

ossf/best-practices-badge#3081 · Aperta il 20/10/2026

I maintainer di solito rispondono entro 1 giorno

Altre issue su Ruby

Parallel `bundle install` races on a git cache shared by multiple gems from one repo

Qualcosa non va?

Valutazione

Direzione di ricerca

Descrizione

Describe the problem as clearly as you can

Did you try upgrading rubygems & bundler?

Post steps to reproduce the problem

Which command did you run?

What were you expecting to happen?

What happened instead?

If not included with the output of your command, run bundle env and paste the output below

Environment

Bundler Build Metadata

Bundler settings

Gemfile

Gemfile

Gemfile.lock

Preparare l'ambiente

Come iniziare

Altre issue di ruby/rubygems

Qualcosa non va?

Qualcosa non va?

Qualcosa non va?

Qualcosa non va?

Qualcosa non va?

Issue simili

Qualcosa non va?

Qualcosa non va?

Qualcosa non va?

Qualcosa non va?

Qualcosa non va?

Ricevi le nuove issue nella tua casella

If not included with the output of your command, run `bundle env` and paste the output below