Per-invocation git subprocess fan-out: 15-25 spawns make wt list/remove/completion cost 4-8s on machines with per-exec endpoint-security tax

Open
  • Is something wrong?
#4,514 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
git, rust
Domain
cli, developer-experience, performance

Research direction

Start by tracing the wt list discovery and Git subprocess calls using the WORKTRUNK_VERBOSE=2 trace described in the issue. Compare which read-only queries can be batched or run concurrently, and check how shell completion uses repository discovery. Done means reducing subprocess spawns or avoiding unnecessary discovery while preserving the reported list, remove, and completion behavior.

Written by the indexing model from the issue text.

Description

Environment

  • macOS 26.6.2, Apple M2 Pro
  • wt v0.80.0 (Homebrew), git 2.56.0
  • Corporate endpoint security (SentinelOne) taxes every process exec ~0.25-0.3s, and execs serialize machine-wide: 20 concurrent git --version spawns take 5.3s wall time here (expected: under 0.5s)

What happens

wt list --format json on a single small repo spawns this sequence (traced with WORKTRUNK_VERBOSE=2, .git/wt/logs/trace.jsonl):

git config --list -z
git --version
git rev-parse --git-common-dir --is-inside-work-tree --show-toplevel --git-dir --symbolic-full-name HEAD
git rev-parse --show-toplevel
git worktree list --porcelain
git for-each-ref ... refs/heads/
git for-each-ref ... refs/remotes/
git log --no-walk ...
git sparse-checkout list
git --no-optional-locks status --porcelain --untracked-files=normal
git ls-files --others --exclude-standard -z
git add -u --sparse
git add --intent-to-add --sparse --pathspec-from-file=- --pathspec-file-nul
git write-tree
git diff-index ... (twice)

That is 15-25 spawns per invocation, mostly sequential phases. With the per-exec tax this is a hard floor:

command wall time
wt --version 0.26s
wt list --format json (one small repo) 4-5s best case; up to 47s under load
wt remove ~30s
shell completion (the clap dynamic completer runs the same discovery) 4-8s per call

Under concurrency it collapses completely. I run a dashboard that polls wt list --format json per known repo (~10 repos), and while that runs, every interactive wt command takes 20-60s, because every new spawn queues behind the concurrent ones.

Ask

Reduce the spawn count per invocation. Some ideas:

  1. Batch the read-only queries: for-each-ref accepts both refs/heads/ and refs/remotes/ in one call; the git --version capability check could be cached (it is constant between upgrades).
  2. Cache the expensive collection (status, divergence) across invocations with a short TTL, or offer a light list mode that skips dirty-status collection. A dashboard caller often needs only branches, paths, and worktree state.
  3. The trace shows long gaps between phases, so stages appear to depend on each other even when the queries are independent. If WORKTRUNK_MAX_CONCURRENT_COMMANDS bounds this, a higher default (or running the independent collection phases concurrently) would cut wall time on machines without the serialized tax.
  4. Shell completion only needs subcommand or branch names, but currently pays the full repo discovery.

Related, from the dashboard side: https://github.com/luiul/understory/issues/9

Happy to attach the full trace.jsonl if useful.

Dominant language
Rust
Stars
8.8k
Forks
315
Avg merge
1d 6h
Merged PRs (30d)
204

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from max-sixty/worktrunk

All issues in max-sixty/worktrunk

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.