引用实现收敛：评估源索引复用与能力边界 / Evaluate source-index reuse and capability boundaries

Start with RFC 0026 and TypedRowSource::reference_source_exists, then review the existing business and recovery tests referenced by the issue. Compare an existing matching source index with the no-index path, covering Exact/Optional, None, ordering, composite paths, predicates, shared ownership, drop and fallback behavior. Done means a bounded prototype or design with measured costs, compatibility rules, and a documented reuse-or-retain decision.

中文说明

目标与当前状态

评估引用实现能否进一步复用普通索引，降低重复状态、检查与迁移成本。保留用户需要的 ADT、嵌套路径、复合 key 和可选负责人语义；不以删除已有业务能力代替实现收敛。正式 v0.13.2 不含引用；main 已支持 legacy storage 12/13 与 native format 14 typed_references 能力，默认仍为 10，采用成本和格式矩阵仍需收敛。

已完成：

• #445 把普通索引与引用 posting 的双向完整性检查收敛到 PostingDefinition，共用 check / migration Ready 遍历。
• #451 移除每条引用的 resident target root。内存和 bounded redb 共用 final-batch validator，目标存在性复用绑定的主键/unique index。reference.rs 从 1076 行减至 1052 行（其中 757 行位于测试模块之前，295 行为测试模块）；这不是 RSS/吞吐测量。
• RFC 0026 提供 source-index 复用的结构成本比较；#449 的恢复旅程和 #452 的真实 TCP/HTTP 旅程证明保留范围的业务用途。#453 的内置文档使规则可发现。

问题 1 的结论：哪些可以直接复用

物理 secondary_index 表、typed key codec、range cursor、增量事务和 generation 生命周期已经复用。Exact 模式且路径顺序/类型一致、全 ascending、无 predicate 的已有 source 索引，是可评估的直接候选；不能假定任意普通索引都覆盖约束。

Optional 模式的 reference key 是投影后的目标类型，None 不产生 posting；普通 source 索引按完整源类型编码并包含 None。当前 TypedRowSource::reference_source_exists 只接收 reference ID、投影后的 encoded target key 和 excluded RowIds。复用需要明确边界适配/接口，而非把 index ID 简单替换过去。降序、复合组件、Exact None 与部分索引还必须分别保持正确。

复用现有用户索引还需要约束其生命周期：drop 时是禁止、原子建立 fallback，还是重新绑定？多个引用共享时如何拥有 backing state？这些规则必须覆盖同一脚本、migration、恢复和回滚。若持久 posting 从 reference ID 转到普通 index ID，旧 reader 不认识 provider，必须有明确拒绝旧读写器的版本/能力边界；不能原样修改 format 12/13 而让旧 reader 查不到 source posting 并放过 restrict。仅复用表不消除约束 catalog 的版本需求。

2026-10-04 有界原型证据（生产复用未完成）

隔离 native 实验已有同二进制成本、固定 backing/catalog 与恢复、离线转换提交前/后退出和sealed 旧链/新 baseline 验证。debug、10k source/1k target、四组各 10 样本中，匹配索引的 postings 31k→21k、100-row insert index changes 300→200，insert/rebuild p50 约改善 22.2%/20.5%。成本属于 lookup-time 原型，不是随后 pinned 版本的测量；无匹配仍 dedicated。

固定 source ID 不随 rename、晚加索引或同名重建而静默切换；缺失/无效 metadata fail closed，物理 metadata 与 schema identity 分开，增量恢复保留归属。转换保留 RowId、allocator、cursor identity、schema、ledger 和原回执，仅推进一次 sequence。active journal 拒绝直接转换：export/verify/disable 后保留 sealed 旧 archive，转换后以新目录建立 baseline。两个目录不是连续链，新 baseline 完成前有备份覆盖窗口。

PR #482 整理生产候选契约。实验 library 184 passed / 6 ignored，fmt 与严格 all-target clippy 通过。生产 metadata/layout/codec、完整 legacy→native→provider 路径、memory/snapshot 统一、released reader、复杂 ADT/cancellation/corruption 和净维护成本仍待验收；实验不进入 main，不分配正式 capability/codec，不改变默认/版本，不完成本 issue。源码、patch、raw samples 留在本地，只公开汇总。

下一阶段与验收

• 给出一个有界 source-provider 原型/设计，比较已有匹配索引与无索引两种情况；记录 source posting 数、代码增减及写入/restrict/rebuild 的实际成本。
• 覆盖 Exact/Optional、None、方向、复合路径、predicate 拒绝、source index drop/共享和原子 fallback；复用既有业务/恢复测试，不另建重复 fixtures。
• 给出结论：只有共享后的净维护成本下降、语义保持且能力边界清楚，才替换持久 source posting；否则保留当前派生 posting 并明确理由。

当前先保留 dedicated source posting，不新增第二套 source-provider 状态机。源索引复用的决定尚未完成；本 issue 保持打开。通用格式模型由 #439 负责，避免在这里再新增成对格式。独立 mutation explain 已由 #457 完成；整体发布验收仍归 #401。

English Description

Objective and current status

Evaluate further reuse of ordinary indexes to reduce duplicate state, checks and migration work. Preserve ADT/nested-path/composite-key/optional-assignee business semantics rather than replacing implementation convergence with feature removal. Published v0.13.2 excludes references; main supports legacy formats 12/13 and the native format 14 typed_references capability while defaulting to 10. Adoption and format-matrix costs remain relevant.

Completed:

• #445 shares bidirectional ordinary/reference posting checks through PostingDefinition in integrity and migration Ready traversal.
• #451 removes per-reference resident target roots and shares final-batch validation across memory/bounded redb, reusing pinned primary/unique target indexes. reference.rs shrinks from 1076 to 1052 lines: 757 before the test module and 295 in it. This is not measured RSS/throughput.
• RFC 0026 compares structural source-index reuse costs. #449 recovery and #452 real TCP/HTTP journeys substantiate the retained business scope; #453 makes the rules discoverable in bundled docs.

Answer to question 1: direct reuse boundaries

The physical index table, typed codec, range cursor, incremental transaction and generation lifecycle are shared already. An existing unconditional, ascending source index matching ordered paths/full types in Exact mode is a candidate; an arbitrary ordinary index is not necessarily suitable.

Optional references project to target types and omit None, while ordinary source indexes encode full source types and retain None. TypedRowSource::reference_source_exists currently receives a reference ID, projected encoded target key and excluded RowIds. Reuse needs boundary/interface adaptation, not an ID substitution. Descending/composite keys, Exact None and partial-index exclusion require explicit correctness rules.

User-index reuse also needs lifecycle/ownership decisions: reject drop, build an atomic fallback, or rebind; shared references; and behavior across scripts, migration, recovery and rollback. Changing durable posting ownership from reference IDs to index IDs requires a compatibility boundary that makes old readers/writers reject unknown providers. Silently changing formats 12/13 could let an old restrict lookup miss source postings. Table reuse alone cannot eliminate constraint catalog versioning.

Current native prototype evidence (production reuse incomplete)

The isolated native experiment covers same-binary costs, pinned backing/catalog and restores, offline conversion with real pre/post-commit exit, and sealed-old/fresh-new archive journeys. Four debug cases, ten samples each over 10k sources/1k targets, reduce matching-index postings 31k→21k and insert index changes 300→200, with median insert/rebuild improvements ~22.2%/20.5%. These timings measure lookup-time backing, not the later pinned variant; unmatched sources stay dedicated.

Pinned identity survives rename and does not switch on later index addition/re-creation. Missing/invalid metadata fails closed, physical choices stay outside schema identity, and incremental replay preserves backing. Conversion retains RowIds/allocators/cursor identity/schema/ledger/original receipts and advances one sequence. Active journals are refused: export/verify/disable preserves a sealed old archive, then conversion/fresh baseline starts a new chain. The directories are not continuous history, and baseline creation leaves a backup-coverage window.

PR #482 records the candidate contract. Experimental library tests pass (184 passed/six ignored), with formatting and strict all-target clippy. Production metadata/layout/codec, full legacy→native→provider acceptance, unified memory/snapshot handling, released readers, complex ADT/cancellation/corruption and net maintenance costs remain gates. Experimental code stays out of main; no official capability/codec/default/version change or completion claim. Source/patch/raw samples stay local; aggregates are published.

Remaining acceptance

• Produce a bounded source-provider prototype/design comparing existing-index and no-index cases; record posting counts, code changes and measured write/restrict/rebuild costs.
• Cover Exact/Optional/None/direction/composite/predicate rejection, shared ownership, index drop and atomic fallback, reusing current business/recovery tests without duplicate fixtures.
• Replace durable source postings only if net maintenance complexity improves while preserving semantics and an explicit compatibility boundary; otherwise retain derived postings with documented reasons.

Keep current dedicated source postings for now rather than add a second provider state machine. The final reuse decision remains open. #439 owns the general format model, avoiding more paired formats here. Independent mutation explain is completed by #457; #401 retains formal release acceptance.