[RFC]: Bringing `glob` Dependency In-House

未关闭
#11,234 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
35/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
冷清
技术栈
javascript, nodejs
领域
tooling

调研方向

先从提议的 @stdlib/utils/regexp-from-glob 和 @stdlib/fs/glob 软件包布局开始,然后盘点 @stdlib/_tools/pkgs、_tools/static-analysis 和 _tools/lint 下 50+ 个 require('glob') 使用方。运行列出的筛选测试和受影响的 CLI 命令,同时将行为与现有依赖进行比较。完成的标准是:两个软件包都已测试并编写文档,使用方已完成迁移,glob 已移除,并且输出和性能保持等效。

由索引模型根据 Issue 内容生成。

描述

Bringing the glob Dependency In-House

Replacing the external glob npm package with stdlib-native equivalents to eliminate a supply-chain risk and align all production and tooling code with stdlib's quality standards (docs, tests, examples, benchmarks, backward compatibility).

Background

What glob Does

A robust glob implementation in JavaScript that matches files using patterns the shell uses (like stars and question marks). It works by walking the filesystem and applying regex-based matching to paths.
More here: https://github.com/isaacs/node-glob

Its core dependencies
  • minimatch: Converts the glob strings into regular expressions and performs the actual string matching.
  • fs.realpath / inflight / once: Various utilities used under the hood to manage concurrent filesystem calls and path resolutions.
How stdlib uses it
  • 50+ files across: @stdlib/_tools/pkgs/*, @stdlib/_tools/lint/*, @stdlib/_tools/static-analysis/*, and various internal bundling or testing scripts.
  • Only the core discovery functions are used: glob( pattern, opts, clbk ) and glob.sync( pattern, opts ).
  • Primary use cases: Finding package.json files, source .js files, or .c files, while explicitly passing an ignore option array (e.g., ['node_modules/**', '.git/**']) to prevent scanning massive dependency directories.
  • No advanced API usage: None of the files instantiate the Glob class directly as an event emitter or use advanced bash features like brace expansion ({a,b}).

Proposed Changes

The plan creates 2 new packages mirroring stdlib's decomposable architecture, with each package being independently consumable.


Component 1: @stdlib/utils/regexp-from-glob

Replaces the minimatch npm dependency (transitive via glob).

A utility to convert standard shell wildcard patterns into native JavaScript RegExp objects.

Scope: Strictly scoped to the wildcards stdlib actually uses (*, **, ?). It will not support brace expansions or extglobs to remain intentionally minimal and maintain high execution speed.

@stdlib/utils/regexp-from-glob/
├── lib/
│   ├── index.js          # re-export main
│   └── main.js           # regexpFromGlob(str) → RegExp
├── test/
│   └── test.js
├── benchmark/
│   └── benchmark.js
├── docs/
│   ├── repl.txt
│   └── types/
│       ├── index.d.ts
│       └── test.ts
├── examples/
│   └── index.js
├── README.md
└── package.json

API:

var regexpFromGlob = require( '@stdlib/utils/regexp-from-glob' );

var re = regexpFromGlob( '**/*.js' );
// returns RegExp

re.test( 'lib/index.js' ); // => true
re.test( 'lib/index.json' ); // => false

Component 2: @stdlib/fs/glob

The core package — the direct replacement for require('glob').

A filesystem traversal utility that applies the generated glob regexes to discover files. This is the main package all _tools files will switch to.

@stdlib/fs/glob/
├── lib/
│   ├── index.js          # re-export main
│   ├── main.js           # async implementation
│   ├── sync.js           # sync implementation
│   ├── walk.js           # internal DFS/BFS directory walker
│   └── validate.js       # options validation
├── test/
│   ├── test.js           # async tests
│   ├── test.sync.js      # sync tests
│   └── test.walk.js      # directory walker tree-pruning tests
├── benchmark/
│   ├── benchmark.js      # benchmark: async traversal
│   └── benchmark.sync.js # benchmark: sync traversal
├── docs/
│   ├── repl.txt
│   └── types/
│       ├── index.d.ts
│       └── test.ts
├── examples/
│   └── index.js
├── README.md
└── package.json

API (drop-in compatible with current usage):

var glob = require( '@stdlib/fs/glob' );

var opts = {
    'cwd': __dirname,
    'ignore': [ 'node_modules/**' ],
    'realpath': true
};

// Async usage
glob( '**/*.js', opts, function onGlob( error, matches ) {
    if ( error ) {
        console.error( error );
        return;
    }
    console.dir( matches );
});

// Sync usage
var matches = glob.sync( '**/*.js', opts );

Key implementation details:

  • Uses @stdlib/fs/read-dir and standard fs.stat under the hood.
  • Tree-Pruning (Crucial for perf): The internal walk.js algorithm must evaluate the ignore option before descending into a directory.
  • Options support: cwd (defaults to process.cwd()), ignore (array of globs), and realpath (boolean).
  • Returns arrays strictly normalized, preventing duplicates.
  • No external dependencies (uses existing stdlib utilities).

Component 3: Migration — Updating All Consumers

This is the mechanical bulk change, done after the new packages are created and verified.

[MODIFY] All 50+ files using require( 'glob' )

The change is a single-line replacement per file:

-var glob = require( 'glob' );
+var glob = require( '@stdlib/fs/glob' );

Migration strategy (phased to reduce risk):

Phase Scope Files
Phase 1 @stdlib/_tools/pkgs/* (find, deps, clis, etc.) ~15 files
Phase 2 @stdlib/_tools/static-analysis/* (sloc-glob, etc.) ~10 files
Phase 3 @stdlib/_tools/lint/* (filenames, pkg-json, etc.) ~20 files
Phase 4 Remaining bundles and scripts ~10 files

Each phase follows the same process:

  1. Run find + sed to replace require( 'glob' ) → require( '@stdlib/fs/glob' ).
  2. Run existing internal tool tests for the affected script area.
  3. Verify output matches the original glob dependency.
Uninstall the dependancy

After all phases are complete:

npm uninstall glob

Verification Plan

Automated Tests
1. Unit tests for @stdlib/utils/regexp-from-glob
make TESTS_FILTER=".*/utils/regexp-from-glob/.*" test

Tests should cover:

  • Core wildcards (*, **, ?).
  • Escaping behavior for dot ., plus +, parentheses (), and brackets [].
  • Prefix, suffix, and exact match conditions.
2. Unit tests for @stdlib/fs/glob
make TESTS_FILTER=".*/fs/glob/.*" test

Tests should cover:

  • Standard sync and async pattern matching against a mock filesystem.
  • ignore arrays effectively pruning traversal.
  • cwd option shifts the search base.
  • realpath correctly resolves to absolute paths.
  • Error handling (e.g., trying to read restricted directories gracefully).
3. Regression tests for migrated modules

Run the full test suite for each phase to ensure the tools operate as expected:

# Phase 1:
make TESTS_FILTER=".*/_tools/pkgs/.*" test

# Phase 2:
make TESTS_FILTER=".*/_tools/static-analysis/.*" test

# Phase 3:
make TESTS_FILTER=".*/_tools/lint/.*" test

# Phase 4: Full internal tools suite
make test
Manual Verification

After migration, verify the tools perform correctly on the monorepo:

  1. Verify Package Discovery:

    node ./lib/node_modules/@stdlib/_tools/pkgs/find/bin/cli
    

    → Verify: Outputs all valid stdlib sub-packages identically to previous runs, without diving into node_modules.

  2. Verify SLOC execution:

    node ./lib/node_modules/@stdlib/_tools/static-analysis/js/sloc-glob/bin/cli
    

    → Verify: Accurately computes lines of code for the workspace.

  3. Performance Check:
    Time the execution before and after the swap.

    time node ./lib/node_modules/@stdlib/_tools/pkgs/find/bin/cli
    

    → Verify: Our native, tree-pruned walker executes in equivalent or faster time than the external dependency.

主要语言
JavaScript
星标
6k
派生
1.3k
平均合并
1 天 21 小时
30 天内合并 PR
224

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

stdlib-js/stdlib 的其他 Issue

查看 stdlib-js/stdlib 的全部 Issue

相似的 Issue

更多 JavaScript Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。