Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Concurrent cache hits for the same task fail to restore outputs with "File exists (os error 17)"

Aperta
  • Qualcosa non va?
#846 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
node.js, rust
Ambito
build-system

Direzione di ricerca

Start with crates/vt/src/session/cache/archive.rs, especially extract_output_archive, and compare it with the restore flow in crates/vt/src/session/cache/remote.rs. The issue describes both concurrent restores and readers observing files during restore; inspect how local and remote cache hits reach extraction and how files are replaced. Done means concurrent hits complete successfully and readers do not observe missing or partial outputs; the issue asks to agree on lock scope before a PR.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

When two vp run processes get a cache hit for the same task at the same time, both extract the output archive over the workspace. tar-rs opens each file with create_new, and when the file exists it removes it and tries once more (entry.rs#L646-L661, same code in 0.4.46, which vite-plus 1.1.0 ships). The two processes race on every path, and one of them usually stops partway. With vite-plus 1.1.0:

$ node gen.mjs ◉ cache hit, replaying
✗ Cache lookup failed: failed to restore cached outputs from …/node_modules/.vite/task-cache/v18/….tar.zst; the archive may have been deleted or corrupted. Run `vp cache clean` to clear the cache.: failed to unpack `…/out/1017.txt`: failed to unpack `out/1017.txt` into `…/out/1017.txt`: File exists (os error 17)

---
vp run: cache hit, 185ms saved.

The archive is fine and the other process completes the restore, so the files end up correct. In vite-plus 1.1.0 the failed run still exits 0 and counts as a hit (#767). On main, after #770, the same race fails the run with exit 1 and suggests cleaning the cache, though nothing in it is wrong. With vt built from 9ba4cc9:

$ node gen.mjs ◉ cache hit, replaying
✗ Cache restore failed. Run `vt cache clean` to clear the cache: failed to extract the output archive: failed to unpack `…/out/10.txt`: failed to unpack `out/10.txt` into `…/out/10.txt`: File exists (os error 17)

Readers are affected even without a second restore. Every hit removes and recreates each output, including files whose content didn't change, so a process reading those outputs while one vp run restores them can find a file missing or shorter than it should be. #822 mentions this as a cost of running one vp run per package.

Our pre-push hook used to run five vp run scripts in parallel, and three of them start with the same codegen task (vp run i18n:generate && ...). We now finish two of those before starting the rest.

Steps to reproduce

package.json:

{ "name": "vt-restore-race", "private": true, "type": "module", "devDependencies": { "vite-plus": "1.1.0" } }

vite.config.ts:

import { defineConfig } from 'vite-plus'

export default defineConfig({
  run: {
    tasks: {
      gen: {
        command: 'node gen.mjs',
        cache: { input: ['gen.mjs'], output: ['out/**'] },
      },
    },
  },
})

gen.mjs:

import { mkdirSync, writeFileSync } from 'node:fs'
mkdirSync('out', { recursive: true })
for (let i = 0; i < 2000; i++) writeFileSync(`out/${i}.txt`, 'x'.repeat(4096))

pnpm install
node_modules/.bin/vp run gen   # populate the cache
for i in $(seq 1 30); do
  (node_modules/.bin/vp run gen > a$i.log 2>&1; echo "exit $?" >> a$i.log) &
  (node_modules/.bin/vp run gen > b$i.log 2>&1; echo "exit $?" >> b$i.log) &
  wait
done
grep -l 'os error 17' *.log | wc -l   # failed restores
grep -L 'exit 0' *.log | wc -l        # runs with a non-zero exit

With vite-plus 1.1.0 this printed 30 and 0: in every pair one run failed, never both, and every run exited 0. The same loop with vt built from 9ba4cc9 and the task in vite-task.json printed 30 and 30. Node 24.21.0, macOS 27.0.1 arm64. I haven't tried Linux or Windows. The restore path in vite-task has no platform-specific code, though the error may differ on Windows.

Reader check

reader.mjs:

import { readFileSync } from 'node:fs'

const end = Date.now() + 1500
let missing = 0
let short = 0
while (Date.now() < end) {
  for (let i = 0; i < 2000; i += 7) {
    try {
      if (readFileSync(`out/${i}.txt`).length !== 4096) short++
    } catch (e) {
      if (e.code === 'ENOENT') missing++
      else throw e
    }
  }
}
console.log({ missing, short })

node reader.mjs & sleep 0.2; node_modules/.bin/vp run gen; wait

With only one vp run restoring, 5 runs on vite-plus 1.1.0 printed missing between 1 and 6 and short 0. With a debug vt build from 9ba4cc9, which restores more slowly, 3 runs printed missing 2 to 4 and short 1 to 4.

Expected: both runs restore the outputs and exit 0, and outputs don't disappear or show partial content while another process reads them.

The restore path takes no lock. The only explicit lock between processes is db_open.lock, held while the database opens (mod.rs#L295-L313), and extract_output_archive unpacks straight into the workspace root (archive.rs#L56-L66). A lock around extraction of the same local archive would fix this reproducer, and the process that gets it second could skip files that already match. Remote hits need a shared key too: each download gets a new archive file name (remote.rs#L204), even when two processes restore the same task. The reader case needs each file swapped in one step, for example by unpacking to a temporary file next to it and renaming it into place. I'd like to agree on the lock scope before working on a PR.

Lingua principale
Rust
Stelle
473
Fork
44
Merge medio
1g 4h
PR unite (30g)
73

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di voidzero-dev/vite-task

Tutte le issue di voidzero-dev/vite-task

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.