Activation records of MeshWeaver.AI and Markdown.Export were read-locked mid-publication on memex-cloud; ModuleLandingService's fail-closed read dropped both modules

Read “Publishing A File On A Shared Volume” first, then inspect ModuleLandingService, ModuleActivationSidecar.Read/WriteOnce, NoReplaceMove.TryMove, and the NoReplaceMoveRatchetGuard scan. Compare the ReplicaSet images and deployment revisions at the burst, and check whether the fingerprint recurs after the roll. Done means identifying whether an older image or a current-image write path explains the exclusive-copy read failures.

Observed — On the memex-cloud deployment, the module activation records of two modules could not be read whole: {path}/<record> on pod memex-portal-deployment-8597b8c8f4-6pxrt and {path}/<record> on pod memex-portal-deployment-d6c9dd984-kv8ld, each open failing with IOException: The process cannot access the file … because it is being used by another process (the companion incident fingerprint 746d74e4fdeb0d4d carries those first lines from the same instants). ModuleLandingService then dropped each module whole from its read — the designed fail-closed rule, not a second defect: a partial read would change which generation the module runs, so the module is excluded from that read and restored by the next read that sees its records whole.

This exact pair of log lines is documented as previously measured on memex (2026-09-16, pod memex-portal-deployment-7cb6684584-jdw7v) in Publishing A File On A Shared Volume: a record published via the copy fallback of File.Move(staged, target, overwrite: false) exists incomplete and under FileShare.None for the length of the copy, and the reader's open fails exactly this way. That page's fix, NoReplaceMove, removes the copy from every publication it covers — and the page itself names "a mixed-image window" as the residual in which a replica still on an older image publishes records with the old call until the roll completes.

Where and when — One burst, one instant: both module-drop lines fall within ~70 ms of each other and the record-read failures within 0.2 ms, on two different modules, on both replicas of the deployment. The two pods belong to different ReplicaSets (d6c9dd984 vs 8597b8c8f4), so two pod templates were live at once — a roll or mixed-image state was in progress. Two different records failing at the same moment is the shape of one landing wave publishing both modules' records concurrently while the periodic read ran — not a steady fault and not a single corrupted record.

Impact — Two reads, each dropping one module for that read only; the message itself states the next whole read restores the module. Per the convergence design, a proposal is never made from a partial read, so the cost on the set lane is a refused proposal for that wave. Not established: whether either affected read was a boot read — at boot the answer is the process's for its lifetime (the pod runs without that module until restarted and reports it as pending). If either was a boot read the impact is materially higher and this should be re-ruled.

Not established — Which process held each record open (writer identity and image version); whether the affected reads were boot reads or {path} reads; whether the two records were subsequently published whole; whether the burst recurs; which landing lane was publishing at that moment (the auto-update wave, an interactive install, or the registry publish endpoint — all write activation records).

Candidate causes and the reading that separates them

1. Mixed-image window (the documented residual). One ReplicaSet runs an image predating NoReplaceMove; its landing publishes activation records through the copy fallback, holding the final name exclusively for the copy's duration. Predicts: bursts coincide with landing waves, and stop once every replica runs the newer image. Separating reading: the {path} of each ReplicaSet's pods at the burst (deployment revision, boot lines), and whether the writer was a pod of the older ReplicaSet.
2. A current-image write path that still reaches an exclusive copy. A File.Move site outside the ratchet guard's scan, or a fallback in one of the covered writers. Predicts: recurrence after the roll completes with both ReplicaSets on the same image. Separating reading: the same image check, plus whether the fingerprint fires again once the roll is done.

One constraint supports a portal-pod writer either way: the deployment's azurefile-memex mount carries nobrl, so the exclusive flock is local to one node's kernel — a reader that sees "being used by another process" shares a node with the holder, which fits a co-located portal replica's landing service and does not fit an arbitrary off-node process.

Where to look — MeshWeaver.PluginCatalog.ModuleLandingService (the log site; the record-read line names the exact record path per module, the module-drop line is its consequence), ModuleActivationSidecar.Read / WriteOnce (the {path}/<Name>/… records), MeshWeaver.Utils.NoReplaceMove.TryMove and the NoReplaceMoveRatchetGuard scan. Docs: Publishing A File On A Shared Volume (the measured 2026-09-16 occurrence of this exact pair, and the mixed-image window), Module Activation Head Ownership (the fail-closed rule), Module Set Convergence (what a refused proposal costs).

Known shape. This is a recurrence of the documented 2026-09-16 signature, not a new defect; the most likely reading is the rollout residual that page already names. If the deployment revision at the burst shows an in-progress roll, a human may close this quickly against that record.

Evidence

2026-10-30 15:48:40Z memex-portal-deployment-d6c9dd984-kv8ld fail: MeshWeaver.PluginCatalog.ModuleLandingService[0]
      Module 'MeshWeaver.Markdown.Export' is NOT loaded from this read: 1 of its records could not be read. A partial read would change which generation it runs — or whether it runs at all — so it is dropped whole, exactly as an unreadable activation entry always was; the next read that sees its records whole restores it.
2026-10-30 15:48:40Z memex-portal-deployment-8597b8c8f4-6pxrt fail: MeshWeaver.PluginCatalog.ModuleLandingService[0]
      Module 'MeshWeaver.AI' is NOT loaded from this read: 1 of its records could not be read. A partial read would change which generation it runs — or whether it runs at all — so it is dropped whole, exactly as an unreadable activation entry always was; the next read that sees its records whole restores it.

This log site's previous ticket, https://github.com/Systemorph/MeshWeaver/issues/6192, was closed about a different fingerprint, and this fault fired after that close. It is left closed: an occurrence of this fingerprint is no evidence that the fault it was closed for came back. This issue now tracks the log site, and later fingerprints of it fold in here.

_{Opened automatically from Admin/_LogIncident/de924b4549cd1052. Recurrences are folded into this issue rather than opening new ones.
It also stands for the whole log site 32129e84e1e00497: other fingerprints of this site fold in here as comments rather than opening tickets of their own.}