Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

Concurrent cache hits for the same task fail to restore outputs with "File exists (os error 17)"

Offen
  • Stimmt etwas nicht?
#846 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
48/100
Issue-Typ
Bug
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
node.js, rust
Bereich
build-system

Rechercherichtung

Start with crates/vt/src/session/cache/archive.rs, especially extract_output_archive, and compare it with the restore flow in crates/vt/src/session/cache/remote.rs. The issue describes both concurrent restores and readers observing files during restore; inspect how local and remote cache hits reach extraction and how files are replaced. Done means concurrent hits complete successfully and readers do not observe missing or partial outputs; the issue asks to agree on lock scope before a PR.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

When two vp run processes get a cache hit for the same task at the same time, both extract the output archive over the workspace. tar-rs opens each file with create_new, and when the file exists it removes it and tries once more (entry.rs#L646-L661, same code in 0.4.46, which vite-plus 1.1.0 ships). The two processes race on every path, and one of them usually stops partway. With vite-plus 1.1.0:

$ node gen.mjs ◉ cache hit, replaying
✗ Cache lookup failed: failed to restore cached outputs from …/node_modules/.vite/task-cache/v18/….tar.zst; the archive may have been deleted or corrupted. Run `vp cache clean` to clear the cache.: failed to unpack `…/out/1017.txt`: failed to unpack `out/1017.txt` into `…/out/1017.txt`: File exists (os error 17)

---
vp run: cache hit, 185ms saved.

The archive is fine and the other process completes the restore, so the files end up correct. In vite-plus 1.1.0 the failed run still exits 0 and counts as a hit (#767). On main, after #770, the same race fails the run with exit 1 and suggests cleaning the cache, though nothing in it is wrong. With vt built from 9ba4cc9:

$ node gen.mjs ◉ cache hit, replaying
✗ Cache restore failed. Run `vt cache clean` to clear the cache: failed to extract the output archive: failed to unpack `…/out/10.txt`: failed to unpack `out/10.txt` into `…/out/10.txt`: File exists (os error 17)

Readers are affected even without a second restore. Every hit removes and recreates each output, including files whose content didn't change, so a process reading those outputs while one vp run restores them can find a file missing or shorter than it should be. #822 mentions this as a cost of running one vp run per package.

Our pre-push hook used to run five vp run scripts in parallel, and three of them start with the same codegen task (vp run i18n:generate && ...). We now finish two of those before starting the rest.

Steps to reproduce

package.json:

{ "name": "vt-restore-race", "private": true, "type": "module", "devDependencies": { "vite-plus": "1.1.0" } }

vite.config.ts:

import { defineConfig } from 'vite-plus'

export default defineConfig({
  run: {
    tasks: {
      gen: {
        command: 'node gen.mjs',
        cache: { input: ['gen.mjs'], output: ['out/**'] },
      },
    },
  },
})

gen.mjs:

import { mkdirSync, writeFileSync } from 'node:fs'
mkdirSync('out', { recursive: true })
for (let i = 0; i < 2000; i++) writeFileSync(`out/${i}.txt`, 'x'.repeat(4096))

pnpm install
node_modules/.bin/vp run gen   # populate the cache
for i in $(seq 1 30); do
  (node_modules/.bin/vp run gen > a$i.log 2>&1; echo "exit $?" >> a$i.log) &
  (node_modules/.bin/vp run gen > b$i.log 2>&1; echo "exit $?" >> b$i.log) &
  wait
done
grep -l 'os error 17' *.log | wc -l   # failed restores
grep -L 'exit 0' *.log | wc -l        # runs with a non-zero exit

With vite-plus 1.1.0 this printed 30 and 0: in every pair one run failed, never both, and every run exited 0. The same loop with vt built from 9ba4cc9 and the task in vite-task.json printed 30 and 30. Node 24.21.0, macOS 27.0.1 arm64. I haven't tried Linux or Windows. The restore path in vite-task has no platform-specific code, though the error may differ on Windows.

Reader check

reader.mjs:

import { readFileSync } from 'node:fs'

const end = Date.now() + 1500
let missing = 0
let short = 0
while (Date.now() < end) {
  for (let i = 0; i < 2000; i += 7) {
    try {
      if (readFileSync(`out/${i}.txt`).length !== 4096) short++
    } catch (e) {
      if (e.code === 'ENOENT') missing++
      else throw e
    }
  }
}
console.log({ missing, short })

node reader.mjs & sleep 0.2; node_modules/.bin/vp run gen; wait

With only one vp run restoring, 5 runs on vite-plus 1.1.0 printed missing between 1 and 6 and short 0. With a debug vt build from 9ba4cc9, which restores more slowly, 3 runs printed missing 2 to 4 and short 1 to 4.

Expected: both runs restore the outputs and exit 0, and outputs don't disappear or show partial content while another process reads them.

The restore path takes no lock. The only explicit lock between processes is db_open.lock, held while the database opens (mod.rs#L295-L313), and extract_output_archive unpacks straight into the workspace root (archive.rs#L56-L66). A lock around extraction of the same local archive would fix this reproducer, and the process that gets it second could skip files that already match. Remote hits need a shared key too: each download gets a new archive file name (remote.rs#L204), even when two processes restore the same task. The reader case needs each file swapped in one step, for example by unpacking to a temporary file next to it and renaming it into place. I'd like to agree on the lock scope before working on a PR.

Vorherrschende Sprache
Rust
Sterne
473
Forks
44
Ø Merge
1 T. 4 Std.
Gemergte PRs (30 T.)
73

Entwicklungsumgebung

In Codespaces öffnen

Startet den Dev-Container des Projekts im Browser, mit Ihrem eigenen GitHub-Konto.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus voidzero-dev/vite-task

Alle Issues in voidzero-dev/vite-task

Ähnliche Issues

Weitere Issues zu Rust

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.