Replay line: a printed replay runs no test under a project config (vitest run, no --config)

Open
#108 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
58/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
javascript
Domain
testing

Research direction

Start with packages/oracles/src/oracle-property.ts and trace how the random unit builds its Replay: line, including rerunOf and the project configuration. Compare the printed command with the working pnpm exec vitest --run --config oracle-gaps/vitest.config.mjs gap-report-oracle command. Done means the printed replay command loads and reruns the failing counterexample; the maintainer still needs to choose how projects supply that command.

Written by the indexing model from the issue text.

Description

Summary

A failing oracle unit prints a replay line that does not rerun the unit when the oracle runs under a project config. In the gap report's package, the printed command is vitest run -t '...'. A bare vitest run uses the package's default config, which finds no file, so every test is skipped and the replay passes vacuously (0 tests run). The unit can be rerun only with the project's own command: pnpm exec vitest --run --config oracle-gaps/vitest.config.mjs gap-report-oracle -t '...'.

P4 (register): a printed replay must rerun the counterexample. This one does not.

Where

packages/oracles/src/oracle-property.ts, the Replay: line built in the random unit's failure (${oracleEnvName('SEED')}=... vitest run ${rerunOf(replayed)}). It always says vitest run, with no config, file, or project command.

Evidence

  • #102: random seed: generated, seed 333199054, path 1:1:1:1. The printed Replay line (CI run 43688703700): ORACLE_SEED=333199054 ORACLE_PATH=1:1:1:1 ORACLE_PROPERTY='oracle-gaps.report' ORACLE_UNIT='random seed: generated' vitest run -t '...'. Run as printed, it ran no test. With the project command it reproduces.
  • #107: the same defect with random seed: a declaration fault: brokenOwner.

Proposed fix

The oracle's configuration should accept the project's replay command (the command that loads the oracle file, such as the --config and file names), and the Replay: line should print it. Until then, the line should at least name the file. Decision for the maintainer on the configuration shape.

Dominant language
JavaScript
Stars
5
Forks
1
Avg merge
6h
Merged PRs (30d)
56

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from TanStack/oracles

All issues in TanStack/oracles

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.