WSL 2: reduce native helper spawns on loads that actually decrypt

Open
  • Is something wrong?
#1,081 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Quiet
Tech stack
rust, typescript
Domain
backend, performance, security

Research direction

Start with decryptValue() and waitForWindowsDaemonFromWsl() in packages/varlock/src/lib/local-encrypt/index.ts, then review executeBatch() in builtin-resolver.ts and the cache encryption path in packages/varlock/src/lib/cache/cache-store.ts. The issue proposes several independent spawn reductions; measure helper spawn counts and load time on WSL before and after each change. Done means multi-value decrypt loads use fewer helper launches while existing decryption behavior remains intact.

Written by the indexing model from the issue text.

Description

Follow-up to #1079, which stopped varlock from spawning the native encryption helper on loads that never touch crypto. This issue covers the remaining case: loads that do need the helper.

Why this matters

On WSL 2 the helper is a Windows .exe launched through interop. In #1078 a single spawn measured ~5s (0.7s load -> 5.9s). Everything below is about spawn count, plus one timeout that is almost certainly wrong at that latency.

1. The pre-decrypt ping timeout is too short (likely a live bug)

decryptValue() pings the daemon with a 1500ms spawnSync timeout before decrypting:

https://github.com/dmno-dev/varlock/blob/80a7c892/packages/varlock/src/lib/local-encrypt/index.ts#L544

If a bare spawn costs ~5s, that ping always times out, even against a warm daemon. daemonAlreadyReady comes back false, so we fall into tryPrestartWindowsDaemonFromWsl() (a wslpath call plus a full powershell.exe -Command Start-Process) and then waitForWindowsDaemonFromWsl(). Worst case for a single decrypt against an already-running daemon:

status spawn + failed ping + wslpath + powershell + ~6 more failed pings + the actual decrypt

Simplest fix: drop the pre-ping entirely and let the first --via-daemon call fail into the bridge-and-retry path that already exists (index.ts:570-576). That removes a spawn from the happy path and removes the false negative.

2. waitForWindowsDaemonFromWsl is a spawn storm, not a poll

https://github.com/dmno-dev/varlock/blob/80a7c892/packages/varlock/src/lib/local-encrypt/index.ts#L242-L253

It sleeps 150ms between pings, but each ping is a full .exe launch, so the interval is noise. Either add a blocking wait-daemon --timeout-ms N command to the Rust helper (one spawn instead of N) or switch to exponential backoff. Separately, the Atomics.wait there blocks the event loop.

3. Batch decrypt in a single spawn (biggest win)

executeBatch() already collects every varlock() call into one batch, then decrypts them sequentially, one .exe per value:

https://github.com/dmno-dev/varlock/blob/80a7c892/packages/varlock/src/lib/local-encrypt/builtin-resolver.ts#L85-L89

Ten encrypted vars means ten spawns. Proposal: add decrypt-batch --data-stdin --via-daemon to the Rust helper, taking { ttyId, items: [{ data, keyId }] } and returning an array of results. The daemon already holds the unlocked key, so it is one biometric gate either way. On macOS this is a modest win (persistent socket, N cheap IPC round trips); on WSL and Windows it is the difference between usable and not.

4. The disk cache has the same shape

https://github.com/dmno-dev/varlock/blob/80a7c892/packages/varlock/src/lib/cache/cache-store.ts#L368-L370

Per-entry encryption means N cached values is N spawns per load. Either route them through the batch command from (3), or wrap the whole cache file under a single envelope (one encrypt/decrypt per load) and keep the per-entry k binding inside the plaintext so entries still cannot be swapped.

5. Persist the status result across invocations

Called out in #1079 as a possible follow-up. Key it on binary path plus mtime plus version, with a short TTL.

Guardrail required first. key_store::generate_key overwrites unconditionally:

https://github.com/dmno-dev/varlock/blob/80a7c892/packages/encryption-binary-rust/src/key_store/mod.rs#L417

and ensureKey() -> keyExists() -> cachedStatusKeys. A stale cached key list that reports "no key" would silently destroy the private key and make every existing ciphertext permanently undecryptable. Before caching status on disk:

  • make generate-key refuse to overwrite an existing key without an explicit --force
  • treat a cached keys list as advisory only, falling through to a fresh probe on any decrypt failure

The first bullet is worth doing on its own merits, independent of caching.

6. Worth measuring: launch the helper from an NTFS path

The .exe lives in node_modules on ext4, so every interop launch loads the PE image over \\wsl.localhost (9p/virtiofs). Copying it once to something like %LOCALAPPDATA%\varlock\bin\varlock-local-encrypt-<version>.exe and launching from there may cut per-spawn cost across the board. This is a hypothesis, not a measured result, but if a spawn really is ~5s then image load is a plausible share of it and it is cheap to test. Needs versioning and an integrity check on the copy.

7. Longer term: reach the daemon without a spawn

Today every WSL operation pays a process launch purely to reach a Windows named pipe, since the WSL side cannot connect to one directly. If the daemon also listened on Windows loopback TCP, the WSL process could talk to it with zero spawns for all operations, not just decrypt. This needs a real auth story: a loopback port has no pipe ACL, so it would need a token file plus session binding. Design task, not a quick fix.

Suggested order

(1) and (3) are independent of each other and of #1079, and together take a multi-decrypt load on WSL from roughly N x 5s down to about two spawns. Those first, then (2), then (5) with its guardrail, then measure (6).

Dominant language
TypeScript
Stars
4.9k
Forks
140
Avg merge
2d 3h
Merged PRs (30d)
35

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from dmno-dev/varlock

All issues in dmno-dev/varlock

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.