requires: declare commands that must resolve, and report where each landed
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 52/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- typescript
- Domain
- cli, testing-qa
Research direction
Start by tracing how front matter handles path: and how the composed PATH reaches sessions, then inspect the existing env: expansion behavior and golden-test runner. Add requires: resolution before the first session, honoring PATHEXT and naming searched directories on failure. Report each resolved command, its path, file count, and session count so a run shows what it actually exercised.
Written by the indexing model from the issue text.
Description
tryscript resolves a command through PATH and says nothing about where it landed. For a
golden suite, that is the difference between a proven result and a green check.
A golden test's whole claim is that a particular program produced particular bytes.
path: prepends to the inherited PATH rather than replacing it, so if a declared entry
fails to resolve — variable unset, build directory cleaned, a typo, a cross-compiled
layout — lookup continues into the inherited PATH and finds whatever is installed there.
The suite then passes while exercising a build nobody selected, with no diagnostic.
That is not hypothetical. It happened in fdu: every golden selected its build with
path: - $TRYSCRIPT_GIT_ROOT/target/debug, and whenever that failed to resolve the
sessions silently fell through to ~/.cargo/bin/fdu — the developer's installed copy —
and passed.
Proposal
requires:
- fdu
Named commands must resolve before the first session runs. If one does not, abort with a
message naming the command and the directories searched, rather than letting a hundred
confusing diffs appear later.
And report where each landed:
resolved fdu -> /…/target/debug/fdu (12 files, 129 sessions)
The reporting half is the one that matters. It makes a run legible rather than merely
correct — the same reason a good test harness says what it ran, not just whether it
passed. Today the only way to know which binary a suite exercised is to reason about
PATH composition and hope.
Notes
- Lookup should respect
PATHEXTon Windows rather than assuming a bare name. - Resolution should use the same composed
PATHthe sessions get, includingpath:
entries, so it answers the question the sessions will actually ask. - Worth failing rather than warning: a suite that runs without the program it is about
does not produce a weaker result, it produces a meaningless one.
Context
fdu worked around this by naming the binary's directory through a variable and
preflighting it in an external runner script before invoking tryscript. That works, but it
is machinery every project would otherwise have to build for itself, and it cannot report
what tryscript itself resolved.
Related: jlevy/tryscript#51, which made env: expand variables the way path: already
did — the same family of "front matter should be able to say what it means about which
program runs".
- Dominant language
- TypeScript
- Stars
- 13
- Forks
- 1
- Avg merge
- 3h 38m
- Merged PRs (30d)
- 3
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from jlevy/tryscript
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 4/5 3-5 days Newbie friendliness 52/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
Similar issues
-
VerificationGate: ATTRIBUTION quote guard never matches a normal quotation (\b around the quote) Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
danielmiessler/LifeOS#2234 ·
-
T: Bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
Mend: dependency security vulnerability untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 70/100