artifact: refresh manifest at runtime without restart (atomic snapshot + auto-refresh)

Đang mở
#96 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
48/100
Loại issue
Tính năng
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Ít trao đổi
Công nghệ
go
Lĩnh vực
backend

Hướng nghiên cứu

Start with artifact/registry.go and artifact/manifest.go, tracing how LoadManifest and RegisterFromConfig populate the manifest table and how Get/Latest read it. Then locate the existing registry tests and run them before making changes. Done when Reload validates a fresh table and preserves the last-good snapshot on errors, readers use an atomic snapshot, and the opt-in poller stops on context cancellation and logs failures.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Type/New Feature

Problem

The artifact registry reads the manifest (manifest.json) exactly once, at startup: LoadManifest → RegisterFromConfig populates the registry's in-memory manifest table, which is then treated as read-only for the lifetime of the process (see artifact/registry.go, artifact/manifest.go).

As a result, any change to the manifest table — adding a new artifact row, or changing a version/path — is invisible until the process is restarted.

Note this is narrower than "the manifest isn't synced":

  • Artifact content is already live. loadAndParse calls loader.Load(...) on every Get/Latest, so editing the bytes of an already-registered artifact is picked up without a restart.
  • The manifest table (id/kind/version → path) is frozen at startup. Only new/changed rows require a restart.

For backends where artifacts are expected to evolve independently of binary deploys (e.g. a repo- or bucket-backed registry), requiring a restart to pick up a new manifest row undercuts the point of an external, versioned registry — the artifact lifecycle should be decoupled from the deployment lifecycle.

Proposed Solution

Make the manifest table refreshable at runtime, without blocking readers and without pushing any new responsibility onto individual loaders (the refresh policy lives in the registry, above the Loader interface).

  1. Atomic snapshot swap (foundation). Move the manifest table behind an atomic.Pointer[snapshot] (sync/atomic, available on the repo's Go version). Readers on the hot path (Get/Latest) do a single lock-free atomic load. A snapshot is immutable once published.

  2. Reload(ctx) error. Re-fetch manifest.json through the existing loader, build a fresh table, validate every row, and only then atomically swap it in. On any error, return it and keep the last-good snapshot — a transient fetch/parse failure must never blank the catalog.

  3. StartAutoRefresh(ctx, interval). A registry-owned background poller that calls Reload every interval, stopping when ctx is cancelled. interval <= 0 is a no-op (feature off), so it is strictly opt-in. The interval comes from the application's config and is passed in here — it is a registry concern, not a loader concern, so loaders stay unaware of refresh entirely.

Failures during polling are logged (slog) and leave the current snapshot in place.

Why an atomic snapshot instead of a mutex: this is a read-heavy / write-rare workload (many Gets between infrequent refreshes) where the whole table is replaced as a unit and slight staleness is acceptable — the textbook case where a copy-on-write snapshot swap avoids taxing every read with lock overhead.

Out of scope (separate follow-ups):

  • Artifact content caching (versioned paths are immutable and cacheable indefinitely; Latest/unversioned entries want a TTL).
  • Any specific trigger wiring beyond the poller (signal handler, admin endpoint, webhook) — the app can drive Reload directly if it wants those.

Alternatives

  • RWMutex guarding the table. Correct, but blocks readers during the swap and taxes every Get with read-lock overhead to guard against a write that almost never happens. Rejected in favor of the lock-free snapshot.
  • Explicit Reload() only, no poller. Smallest surface, but pushes the "when to refresh" burden onto every consumer. Keeping Reload() public still allows this, so the poller is additive rather than exclusive.
  • Restart / rolling redeploy to pick up manifest changes. The status quo. Acceptable for file-mounted deployments where a manifest change already coincides with a redeploy, but defeats the decoupling for registries backed by an independently-updated source.
  • Per-loader refresh. Would duplicate the same polling logic across every backend and leak a registry-level policy into the Loader interface. Rejected — refresh belongs in the registry.
Ngôn ngữ chính
Go
Star
6
Fork
8
Merge trung bình
21 giờ 36 phút
Pull request đã merge (30 ngày)
42

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của OpenNSW/core

Tất cả issue của OpenNSW/core

Issue tương tự

Thêm issue về Go

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.