Cert hot-reload misses Kubernetes projected-volume (`..data` symlink-swap) rotations

Open
#17 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
rust
Domain
backend, security

Research direction

Start by reading the standalone symphony-server hot-reload implementation introduced in #14, focusing on parent-directory watching, basename filtering, and the debounced reconcile. Compare the proposed symlink-swap handling options and verify the chosen behavior against projected-volume rotations and existing direct-file updates. Done means a Kubernetes-style ..data rotation triggers certificate reload without breaking current filtering or debounce behavior.

Written by the indexing model from the issue text.

Description

Background

@harperfast/[email protected] (#14) added live cert-file hot-reload to the standalone symphony-server: it watches the parent directories of the cert/key files referenced by the config and, on a change event, filters by the referenced basename (certFilesByDir.get(dir)?.has(filename)) before triggering a debounced reconcile.

Limitation

Basename filtering handles in-place writes and renames of the real files — which is what host-manager does (writes keys/fullchain.pem / keys/privkey.pem directly). It does not handle rotation performed by swapping a symlinked directory, which is how several common environments rotate certs:

  • Kubernetes Secret / projected volumes (and cert-manager) update a mount atomically by writing a new timestamped dir (..2026_07_06_...) and re-pointing the ..data symlink. The watched directory sees rename events for ..data / ..2026_*, none of which match the configured basename (e.g. tls.crt), so they're filtered out and no reconcile fires.
  • Let's Encrypt live/ symlinks can hit the same shape depending on how the symlink is replaced.

Net: in a symlink-swap deployment the hot-reload silently no-ops, and symphony keeps serving the old cert until a restart or an unrelated config write. (Per-route KeyMismatch isolation and last-good carry-forward from #14 still hold — this is purely about noticing the rotation.)

Possible fixes

  1. Resolve symlinks (fs.realpathSync) and watch the real target directory, re-resolving on each reconcile.
  2. When the watched dir contains a ..data-style symlink, don't drop unmatched rename events — let them through to the debounced reconcile (it's already coalesced + cheap).
  3. A periodic re-stat fallback (compare mtime/inode of referenced files) as a backstop for watchers that miss events.

Option 2 is the smallest; option 1 is the most general. Both should stay bounded (dedup by dir, existing debounce).

Priority

Low for the current deployment model (host-manager writes real files in place, which works today). Relevant if symphony is ever run under Kubernetes projected volumes / cert-manager.

Filed as a follow-up to #14. Issue text generated by an LLM (Claude Opus 4.8) on Kris's behalf.

Dominant language
TypeScript
Stars
1
Forks
0
Avg merge
11h 49m
Merged PRs (30d)
12

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from HarperFast/symphony

All issues in HarperFast/symphony

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.