[RFC]: Bringing `glob` Dependency In-House
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 35/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 冷清
- 技术栈
- javascript, nodejs
- 领域
- tooling
调研方向
先从提议的 @stdlib/utils/regexp-from-glob 和 @stdlib/fs/glob 软件包布局开始,然后盘点 @stdlib/_tools/pkgs、_tools/static-analysis 和 _tools/lint 下 50+ 个 require('glob') 使用方。运行列出的筛选测试和受影响的 CLI 命令,同时将行为与现有依赖进行比较。完成的标准是:两个软件包都已测试并编写文档,使用方已完成迁移,glob 已移除,并且输出和性能保持等效。
由索引模型根据 Issue 内容生成。
描述
Bringing the glob Dependency In-House
Replacing the external glob npm package with stdlib-native equivalents to eliminate a supply-chain risk and align all production and tooling code with stdlib's quality standards (docs, tests, examples, benchmarks, backward compatibility).
Background
What glob Does
A robust glob implementation in JavaScript that matches files using patterns the shell uses (like stars and question marks). It works by walking the filesystem and applying regex-based matching to paths.
More here: https://github.com/isaacs/node-glob
Its core dependencies
minimatch: Converts the glob strings into regular expressions and performs the actual string matching.fs.realpath/inflight/once: Various utilities used under the hood to manage concurrent filesystem calls and path resolutions.
How stdlib uses it
- 50+ files across:
@stdlib/_tools/pkgs/*,@stdlib/_tools/lint/*,@stdlib/_tools/static-analysis/*, and various internal bundling or testing scripts. - Only the core discovery functions are used:
glob( pattern, opts, clbk )andglob.sync( pattern, opts ). - Primary use cases: Finding
package.jsonfiles, source.jsfiles, or.cfiles, while explicitly passing anignoreoption array (e.g.,['node_modules/**', '.git/**']) to prevent scanning massive dependency directories. - No advanced API usage: None of the files instantiate the
Globclass directly as an event emitter or use advanced bash features like brace expansion ({a,b}).
Proposed Changes
The plan creates 2 new packages mirroring stdlib's decomposable architecture, with each package being independently consumable.
Component 1: @stdlib/utils/regexp-from-glob
Replaces the
minimatchnpm dependency (transitive viaglob).
A utility to convert standard shell wildcard patterns into native JavaScript RegExp objects.
Scope: Strictly scoped to the wildcards stdlib actually uses (*, **, ?). It will not support brace expansions or extglobs to remain intentionally minimal and maintain high execution speed.
@stdlib/utils/regexp-from-glob/
├── lib/
│ ├── index.js # re-export main
│ └── main.js # regexpFromGlob(str) → RegExp
├── test/
│ └── test.js
├── benchmark/
│ └── benchmark.js
├── docs/
│ ├── repl.txt
│ └── types/
│ ├── index.d.ts
│ └── test.ts
├── examples/
│ └── index.js
├── README.md
└── package.json
API:
var regexpFromGlob = require( '@stdlib/utils/regexp-from-glob' );
var re = regexpFromGlob( '**/*.js' );
// returns RegExp
re.test( 'lib/index.js' ); // => true
re.test( 'lib/index.json' ); // => false
Component 2: @stdlib/fs/glob
The core package — the direct replacement for
require('glob').
A filesystem traversal utility that applies the generated glob regexes to discover files. This is the main package all _tools files will switch to.
@stdlib/fs/glob/
├── lib/
│ ├── index.js # re-export main
│ ├── main.js # async implementation
│ ├── sync.js # sync implementation
│ ├── walk.js # internal DFS/BFS directory walker
│ └── validate.js # options validation
├── test/
│ ├── test.js # async tests
│ ├── test.sync.js # sync tests
│ └── test.walk.js # directory walker tree-pruning tests
├── benchmark/
│ ├── benchmark.js # benchmark: async traversal
│ └── benchmark.sync.js # benchmark: sync traversal
├── docs/
│ ├── repl.txt
│ └── types/
│ ├── index.d.ts
│ └── test.ts
├── examples/
│ └── index.js
├── README.md
└── package.json
API (drop-in compatible with current usage):
var glob = require( '@stdlib/fs/glob' );
var opts = {
'cwd': __dirname,
'ignore': [ 'node_modules/**' ],
'realpath': true
};
// Async usage
glob( '**/*.js', opts, function onGlob( error, matches ) {
if ( error ) {
console.error( error );
return;
}
console.dir( matches );
});
// Sync usage
var matches = glob.sync( '**/*.js', opts );
Key implementation details:
- Uses
@stdlib/fs/read-dirand standardfs.statunder the hood. - Tree-Pruning (Crucial for perf): The internal
walk.jsalgorithm must evaluate theignoreoption before descending into a directory. - Options support:
cwd(defaults toprocess.cwd()),ignore(array of globs), andrealpath(boolean). - Returns arrays strictly normalized, preventing duplicates.
- No external dependencies (uses existing
stdlibutilities).
Component 3: Migration — Updating All Consumers
This is the mechanical bulk change, done after the new packages are created and verified.
[MODIFY] All 50+ files using require( 'glob' )
The change is a single-line replacement per file:
-var glob = require( 'glob' );
+var glob = require( '@stdlib/fs/glob' );
Migration strategy (phased to reduce risk):
| Phase | Scope | Files |
|---|---|---|
| Phase 1 | @stdlib/_tools/pkgs/* (find, deps, clis, etc.) |
~15 files |
| Phase 2 | @stdlib/_tools/static-analysis/* (sloc-glob, etc.) |
~10 files |
| Phase 3 | @stdlib/_tools/lint/* (filenames, pkg-json, etc.) |
~20 files |
| Phase 4 | Remaining bundles and scripts | ~10 files |
Each phase follows the same process:
- Run
find+sedto replacerequire( 'glob' )→require( '@stdlib/fs/glob' ). - Run existing internal tool tests for the affected script area.
- Verify output matches the original glob dependency.
Uninstall the dependancy
After all phases are complete:
npm uninstall glob
Verification Plan
Automated Tests
1. Unit tests for @stdlib/utils/regexp-from-glob
make TESTS_FILTER=".*/utils/regexp-from-glob/.*" test
Tests should cover:
- Core wildcards (
*,**,?). - Escaping behavior for dot
., plus+, parentheses(), and brackets[]. - Prefix, suffix, and exact match conditions.
2. Unit tests for @stdlib/fs/glob
make TESTS_FILTER=".*/fs/glob/.*" test
Tests should cover:
- Standard sync and async pattern matching against a mock filesystem.
ignorearrays effectively pruning traversal.cwdoption shifts the search base.realpathcorrectly resolves to absolute paths.- Error handling (e.g., trying to read restricted directories gracefully).
3. Regression tests for migrated modules
Run the full test suite for each phase to ensure the tools operate as expected:
# Phase 1:
make TESTS_FILTER=".*/_tools/pkgs/.*" test
# Phase 2:
make TESTS_FILTER=".*/_tools/static-analysis/.*" test
# Phase 3:
make TESTS_FILTER=".*/_tools/lint/.*" test
# Phase 4: Full internal tools suite
make test
Manual Verification
After migration, verify the tools perform correctly on the monorepo:
-
Verify Package Discovery:
node ./lib/node_modules/@stdlib/_tools/pkgs/find/bin/cli→ Verify: Outputs all valid
stdlibsub-packages identically to previous runs, without diving intonode_modules. -
Verify SLOC execution:
node ./lib/node_modules/@stdlib/_tools/static-analysis/js/sloc-glob/bin/cli→ Verify: Accurately computes lines of code for the workspace.
-
Performance Check:
Time the execution before and after the swap.time node ./lib/node_modules/@stdlib/_tools/pkgs/find/bin/cli→ Verify: Our native, tree-pruned walker executes in equivalent or faster time than the external dependency.
- 主要语言
- JavaScript
- 星标
- 6k
- 派生
- 1.3k
- 平均合并
- 1 天 21 小时
- 30 天内合并 PR
- 224
环境准备
在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
stdlib-js/stdlib 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 72/100
stdlib-js/stdlib#16655 · 1 条评论 ·
维护者通常 1 天内回复
-
Fix JavaScript lint errors可能已有人在做 @protestershankar 于 6 天前认领。 未关闭Good First Issue
难度 2/5 1-3 小时 新手友好度 82/100
stdlib-js/stdlib#16563 · 3 条评论 ·
维护者通常 1 天内回复
-
Fix C lint errors可能已有人在做 @raus7n 于 14 天前认领。 未关闭Good First Issue
难度 1/5 1 小时以内 新手友好度 82/100
stdlib-js/stdlib#16454 · 1 条评论 ·
维护者通常 1 天内回复
-
Fix JavaScript lint errors可能已有人在做 @Devansh-18155 于 39 天前认领。 未关闭Good First Issue
难度 1/5 1 小时以内 新手友好度 82/100
stdlib-js/stdlib#15970 · 1 条评论 ·
维护者通常 1 天内回复
-
Fix JavaScript lint errors可能已有人在做 @lb1192176991-lab 于 44 天前认领。 未关闭Good First Issue
难度 1/5 1 小时以内 新手友好度 90/100
stdlib-js/stdlib#15831 · 2 条评论 ·
维护者通常 1 天内回复
相似的 Issue
-
难度 1/5 1 小时以内 新手友好度 69/100
维护者通常 1 天内回复
-
bug ready-for-agent
难度 2/5 1-3 小时 新手友好度 72/100
nearform/unbreakci#897 · 1 条评论 · 1 个 reaction ·
维护者通常 1 天内回复
-
automation chore ci-cd consistency developer-experience difficulty: easy discoverability documentation enhancement github-actions good first issue help wanted hygiene priority: P3 seo shell yaml
难度 1/5 1 小时以内 新手友好度 76/100
-
bug
难度 2/5 1-3 小时 新手友好度 84/100
swc-project/swc-next-issues#394 ·
-
难度 1/5 1-3 小时 新手友好度 88/100
githubnext/gh-aw-workshop#5790 ·
维护者通常 1 天内回复