test(compare): run the footguns page's js examples against the pinned libraries

Open
#324 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
66/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active

Research direction

Start with docs/explanation/footguns.md, compare/package.json, and the existing compare job that installs the pinned libraries. Check how doctests parse and format examples, then add a runner for the page’s js fences and execute each in a timed child process. Done means CI reports mismatched or uncounted result comments with the fence identified, while matching pinned-library examples pass.

Written by the indexing model from the issue text.

Description

docs/explanation/footguns.md shows other libraries' wrong results in js fences. Each fence starts with a // library version comment and gives the result as // → value or // throws Class: message. The doctests run only ts blocks, so CI never checks these fences.

Every result was checked by hand against the versions pinned in compare/package.json when #322 merged: 27 fences, 37 results, on Node 24. A pin bump could change a library's behaviour and quietly make a claim on the page false.

Proposal: run the page's js fences in the compare job, which already installs those libraries:

  • For each js fence, write the code to a module that resolves imports from compare/node_modules. Map dinero.js to dinero-v1 when the header says 1.9.1.
  • Turn each expr; // → value line into a check that formats the value the way the doctests do: strings in single quotes, arrays as [a, b], everything else with String().
  • Turn each // throws Class: message line into a check that the call throws that class with a message starting with the text given.
  • Run each module in a child process with a timeout. Fail on any mismatch, and when the count of checked comments differs from the count of // → and // throws comments, so that a statement the formatter splits over several lines can't be skipped without anyone noticing.
  • When a pin bump changes a result, the job should fail and name the fence. Then either update the page or move the example to "fixed in X".
Dominant language
TypeScript
Stars
0
Forks
0
Avg merge
35m
Merged PRs (30d)
242

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from public/ducat

All issues in public/ducat

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.