Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

writeSchedule lowers only the writing node's claim floor, but rows are residency-routed to their owner — ~75% of writes can strand rows silently

Đang mở
  • Có gì không ổn?
#111 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
35/100
Loại issue
Lỗi
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
javascript
Lĩnh vực
backend, distributed-systems

Hướng nghiên cứu

Bắt đầu tại writeSchedule và lần theo lowerFloorFor qua leaseTable().lowerFloorTo(...) và coordination.SharedBuffer cục bộ. Sau đó đọc resources/Target.js, Sitemap.js và getResidencyByUrl để so sánh cách xử lý hiện có theo từng URL với quyền sở hữu residency. Done phải bao gồm một hướng đi đã được thống nhất, phạm vi bao phủ cho các thao tác ghi thông thường trong plugin và bằng chứng cho thấy các hàng không bị để lại dưới floor của node sở hữu.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Rewritten 2026-08-19. The first version of this issue blamed replication. That was wrong —
RenderSchedule is not replicated; rows are residency-routed and each key lives on exactly one
node (verified: a 27-key probe returned 1/16/10/0 across four nodes, perfectly disjoint). The
actual mechanism is below, and it is worse, because it fires from ordinary in-plugin writes.

Summary

writeSchedule lowers the claim floor of the node executing the write. But the row it just
wrote is residency-routed to whichever node owns that cacheKey. On a 4-node cluster ~75% of
writes therefore lower a floor belonging to the wrong node, and the owner's floor is never lowered
at all.

Any row filed with a due minute older than queue.claimFloor.guard (default 5 min) lands below
its owner's floor and is never claimed again — silently, permanently, from a fully funnel-routed
write.

The mechanism

export const writeSchedule = async (cacheKey, { nextRenderTime, fromSitemap } = {}) => {
	...
	await scheduleTable().put(cacheKey, { nextRenderTime, fromSitemap });
	lowerFloorFor(nextRenderTime);   // ← local SharedBuffer, i.e. THIS node's floor
};

lowerFloorFor → leaseTable().lowerFloorTo(...) is an Atomics CAS-min against the local
coordination.SharedBuffer (replicate: false, node-local by construction). The put, however,
is residency-routed — and per the module comment on writeSchedule, deliberately does not block
on the owner:

a write to a residency-pinned key this node does not own does NOT block on the owner (measured:
500 writes in 10.7 ms, mean 0.021 ms, against residency pinned to a node that does not exist)

Ownership is getResidencyByUrl → fnv1a32(\${url}|${node}`)` rendezvous hashing, so for any
given writer 3 of 4 keys are owned elsewhere.

This is already documented in the tree

resources/Target.js, in revalidate's phase 2 — the hazard is known and currently mitigated
only by writers remembering to recompute the minute per URL:

THE CURRENT MINUTE, PER URL — never captured once for the whole sweep. Phase 2 writes up to
scan.collectCap × devices rows with a PrerenderedPage.get per key, which at scale takes tens
of minutes. Rows are residency-routed, so ~75% land on nodes whose claim floor this process
cannot lower and which hold it at nowMinute − queue.claimFloor.guard: every row filed with a
minute more than the guard band old lands BELOW the owner's floor and is never claimed again —
silently, from a fully funnel-routed in-plugin write, and permanently where resetInterval: 0.
Sitemap.js already computes it per entry for the same reason.

So the invariant "never let more than guard elapse between computing a due minute and writing
it" is load-bearing across every schedule writer, enforced nowhere, and unobservable when broken.
Two call sites currently carry hand-written comments to remember it.

Impact

Observed on a 4-node production cluster (plugin 0.49.0 / harper-pro 5.2.3). One node accumulated
92,873 rows below its floor across 2,101 distinct due-minutes, oldest due 2026-08-01 — 18 days
of a growing slice of its shard silently not rendering, while the node reported healthy and kept
serving. Peer nodes were at 507 / 3 / 1 overdue.

Downstream, those keys held damaged blobs that could only be replaced by a re-render, so
replication base copies from the affected node latched indefinitely (harper-pro#699 territory) —
a queue bug surfacing as a storage symptom.

Why the nominated recovery does not cover it

queue.claimFloor.resetInterval's docstring names the out-of-plugin write paths it exists to
cover — the operations API and the exported REST surface. Residency-routed in-plugin writes
are not mentioned, and they are the high-volume path. The reset is also worker-0-gated inside
syncQueueState (see the open question in #110).

Possible directions

  1. Lower the owner's floor, not the writer's. The write already routes to the owner; the floor
    lowering needs to ride with it rather than being applied locally. That is the fix that makes
    the invariant unnecessary.
  2. Bound the forward advance so a floor can never outrun rows the pass never scanned (#110).
    That alone would stop the stranding even if the lowering stays local.
  3. Make the below-floor condition observable. The backlog snapshot already computes
    below_floor; a node with essentially its whole due backlog below the floor should alarm.
  4. Failing 1, at minimum assert the invariant at the funnel — reject or warn on a writeSchedule
    whose nextRenderTime minute is already older than guard, so a slow batch fails loudly
    instead of silently stranding rows on a peer.
Ngôn ngữ chính
JavaScript
Star
0
Fork
0
Merge trung bình
8 giờ 25 phút
Pull request đã merge (30 ngày)
71

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của HarperFast/prerender-plugin

Tất cả issue của HarperFast/prerender-plugin

Issue tương tự

Thêm issue về JavaScript

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.