perf(cli): establish built help and version startup baselines

Open
#2,482 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
22/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
cli, nodejs, typescript
Domain
cli, performance

Research direction

Start with scripts/measure-cold-start.mjs and the built CLI's help and version entry points, but the build must first succeed: TypeScript and the locked dependency restore are currently blocked, and dist is absent. Add focused runner tests for missing dist, nonzero child exit, timeout and invalid samples, using deterministic inputs for quantile and output-shape checks. Done means three rounds of at least 20 fresh processes per command with raw observations, SHA, Node/platform details and p50/p95 published, with no performance claim.

Written by the indexing model from the issue text.

Description

Summary

Split the packaged CLI startup measurement out of the broader benchmark request github/squad#2476. This is a measurement-only issue for two commands: the built CLI's help and version entry points. It does not authorize startup optimization or a new benchmark framework.

Parent: github/squad#2481.

Proposed primary owner: cli-dev.

Existing evidence and blocker

At audited revision 47082a81799710c3e457b594123d190c00596ebf, scripts/measure-cold-start.mjs exists, but built CLI startup could not be measured. TypeScript was unavailable, locked dependency restoration returned a configured-feed E404 for hono-4.13.17.tgz, and dist was absent. There is no established startup latency or RSS result yet, and no startup bottleneck/speedup is claimed.

Use the existing built/shipping CLI entry point and runner after the documented restore/build succeeds. Source-loader or SDK-import probes are not substitutes, as illustrated by the packaged-path correction in microsoft/aspire#16704.

Scope

  • In: help/version process-start-to-exit wall time and process maximum RSS, existing runner output/error handling, portable reproduction instructions and raw artifacts.
  • Out: interactive shell/session resume, model calls, config/charter/memory/backend benchmarks, lazy-import changes, CI gates, cross-platform rollout and solving package-feed infrastructure in product code.

Measurable acceptance and tests

  • For each command, collect three rounds of at least 20 fresh child processes on the same recorded SHA/build/environment. Record first observations separately and label OS filesystem-cache state; do not describe warm-filesystem fresh processes as cold storage.
  • Publish every raw elapsed/RSS observation, exit status, exact command, SHA, Node/platform/build mode and per-round p50/p95. RSS must be labeled as process maximum RSS, not allocation volume or total agent-session memory.
  • Run without inference/network workload or user-home mutation. Verify expected help/version output and successful exit on every measured invocation; keep output equivalent to the unmodified built CLI.
  • Focused runner tests prove missing dist, child nonzero exit, timeout and missing/invalid measurement data fail explicitly rather than emit a successful zero/NaN sample. Quantile/output-shape tests use deterministic inputs.
  • Publish a portable command and shared results artifact; no dependency on a private Copilot session path. If restore remains blocked, retain the exact blocker and leave measurement acceptance incomplete.

Done means

A reproducible, correctly labeled baseline for these two commands and failure-sensitive runner tests. No performance improvement is required or claimed by this measurement issue. Any identified startup optimization requires its own narrowly scoped, measured follow-up. The rollup github/squad#2481 tracks this result.

Dominant language
TypeScript
Stars
3.4k
Forks
516
Avg merge
1d 15h
Merged PRs (30d)
116

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from github/squad

All issues in github/squad

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.