Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

Concurrent cache hits for the same task fail to restore outputs with "File exists (os error 17)"

Aberta
  • Algo errado?
#846 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
48/100
Tipo de issue
Bug
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
node.js, rust
Domínio
build-system

Direção de pesquisa

Start with crates/vt/src/session/cache/archive.rs, especially extract_output_archive, and compare it with the restore flow in crates/vt/src/session/cache/remote.rs. The issue describes both concurrent restores and readers observing files during restore; inspect how local and remote cache hits reach extraction and how files are replaced. Done means concurrent hits complete successfully and readers do not observe missing or partial outputs; the issue asks to agree on lock scope before a PR.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

When two vp run processes get a cache hit for the same task at the same time, both extract the output archive over the workspace. tar-rs opens each file with create_new, and when the file exists it removes it and tries once more (entry.rs#L646-L661, same code in 0.4.46, which vite-plus 1.1.0 ships). The two processes race on every path, and one of them usually stops partway. With vite-plus 1.1.0:

$ node gen.mjs ◉ cache hit, replaying
✗ Cache lookup failed: failed to restore cached outputs from …/node_modules/.vite/task-cache/v18/….tar.zst; the archive may have been deleted or corrupted. Run `vp cache clean` to clear the cache.: failed to unpack `…/out/1017.txt`: failed to unpack `out/1017.txt` into `…/out/1017.txt`: File exists (os error 17)

---
vp run: cache hit, 185ms saved.

The archive is fine and the other process completes the restore, so the files end up correct. In vite-plus 1.1.0 the failed run still exits 0 and counts as a hit (#767). On main, after #770, the same race fails the run with exit 1 and suggests cleaning the cache, though nothing in it is wrong. With vt built from 9ba4cc9:

$ node gen.mjs ◉ cache hit, replaying
✗ Cache restore failed. Run `vt cache clean` to clear the cache: failed to extract the output archive: failed to unpack `…/out/10.txt`: failed to unpack `out/10.txt` into `…/out/10.txt`: File exists (os error 17)

Readers are affected even without a second restore. Every hit removes and recreates each output, including files whose content didn't change, so a process reading those outputs while one vp run restores them can find a file missing or shorter than it should be. #822 mentions this as a cost of running one vp run per package.

Our pre-push hook used to run five vp run scripts in parallel, and three of them start with the same codegen task (vp run i18n:generate && ...). We now finish two of those before starting the rest.

Steps to reproduce

package.json:

{ "name": "vt-restore-race", "private": true, "type": "module", "devDependencies": { "vite-plus": "1.1.0" } }

vite.config.ts:

import { defineConfig } from 'vite-plus'

export default defineConfig({
  run: {
    tasks: {
      gen: {
        command: 'node gen.mjs',
        cache: { input: ['gen.mjs'], output: ['out/**'] },
      },
    },
  },
})

gen.mjs:

import { mkdirSync, writeFileSync } from 'node:fs'
mkdirSync('out', { recursive: true })
for (let i = 0; i < 2000; i++) writeFileSync(`out/${i}.txt`, 'x'.repeat(4096))

pnpm install
node_modules/.bin/vp run gen   # populate the cache
for i in $(seq 1 30); do
  (node_modules/.bin/vp run gen > a$i.log 2>&1; echo "exit $?" >> a$i.log) &
  (node_modules/.bin/vp run gen > b$i.log 2>&1; echo "exit $?" >> b$i.log) &
  wait
done
grep -l 'os error 17' *.log | wc -l   # failed restores
grep -L 'exit 0' *.log | wc -l        # runs with a non-zero exit

With vite-plus 1.1.0 this printed 30 and 0: in every pair one run failed, never both, and every run exited 0. The same loop with vt built from 9ba4cc9 and the task in vite-task.json printed 30 and 30. Node 24.21.0, macOS 27.0.1 arm64. I haven't tried Linux or Windows. The restore path in vite-task has no platform-specific code, though the error may differ on Windows.

Reader check

reader.mjs:

import { readFileSync } from 'node:fs'

const end = Date.now() + 1500
let missing = 0
let short = 0
while (Date.now() < end) {
  for (let i = 0; i < 2000; i += 7) {
    try {
      if (readFileSync(`out/${i}.txt`).length !== 4096) short++
    } catch (e) {
      if (e.code === 'ENOENT') missing++
      else throw e
    }
  }
}
console.log({ missing, short })

node reader.mjs & sleep 0.2; node_modules/.bin/vp run gen; wait

With only one vp run restoring, 5 runs on vite-plus 1.1.0 printed missing between 1 and 6 and short 0. With a debug vt build from 9ba4cc9, which restores more slowly, 3 runs printed missing 2 to 4 and short 1 to 4.

Expected: both runs restore the outputs and exit 0, and outputs don't disappear or show partial content while another process reads them.

The restore path takes no lock. The only explicit lock between processes is db_open.lock, held while the database opens (mod.rs#L295-L313), and extract_output_archive unpacks straight into the workspace root (archive.rs#L56-L66). A lock around extraction of the same local archive would fix this reproducer, and the process that gets it second could skip files that already match. Remote hits need a shared key too: each download gets a new archive file name (remote.rs#L204), even when two processes restore the same task. The reader case needs each file swapped in one step, for example by unpacking to a temporary file next to it and renaming it into place. I'd like to agree on the lock scope before working on a PR.

Linguagem predominante
Rust
Estrelas
473
Forks
44
Merge médio
1d 4h
PRs com merge (30d)
73

Preparar o ambiente

Abrir no Codespaces

Inicia o contêiner de desenvolvimento do projeto no navegador, com a sua própria conta do GitHub.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de voidzero-dev/vite-task

Todas as issues de voidzero-dev/vite-task

Issues semelhantes

Mais issues de Rust

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.