CDS transport fault taxonomy, chunked live read with adjacency verification, and an asymmetric retry policy

Review cds_transport.py, the #5 fakes, and spec §4.3 alongside the linked plan before choosing an implementation slice. Done means both transports share the listed fault hierarchy, stream emits verified-adjacent chunks, passive retries are bounded while excited records do not retry, and the two cds settings are validated.

Spin-out of #32 · Stage C (#11, #12) · spec §4.3.1, §4.3.2, §4.3.3, §4.3.4, §4.3.7.

Covers #32 items 1, 3, 8, 10.

Only one thing in this issue has been observed (see "the weak empirical driver" below). The rest
is architecture for the multi-user, unattended case. Filed now because this is the single most
expensive thing to retrofit in the whole campaign.

1. The fault taxonomy — the highest-value item

cds_transport.py (#11) must define one exception hierarchy, and both AWGNDSTransport and
TwinTransport must raise the same types, so every handler downstream is testable on the #5 fakes:

CDSTransportError            (base — catching this alone is enough to stop driving)
    TransportUnavailable     network error, framebuilder gone            #32 items 3, 10
    TestpointLost            slot cleared by another user/process        #32 items 1, 2
    TestpointTimeout         allocation did not complete                 #32 item 8
    ChannelNotFound          pre-flight existence                        #32 item 7
    ChannelNotInjectable     slow read-only EPICS as an excitation ch.   #32 item 6
    DataIntegrityError       non-finite / short / drive mismatch         #32 item 5
    TimingFault              start_time, sample count, rate, GPS         #32 item 9

If the transports raise bare RuntimeError, every handler added later is a retrofit across the whole
backend. Two standing rules: an unclassified fault still hits the base and therefore still
triggers safe teardown — the default is safe; and a fault with no matching class is a gap in the
taxonomy, not a licence for an ad-hoc handler — add the class.

The taxonomy is designed to absorb faults not listed in #32. Place a new one by answering two
questions: (i) can we get data at all? no → transport; data arrives but is wrong → integrity; data
is fine but the plant never received the drive → hardware-state. (ii) is the channel recorded or a
test point? That decides whether a retry is even meaningful. Worked examples in spec §4.3.4 — a
front-end model restart, an excitation slot taken by another client, ADC/DAC overflow, framebuilder
disk full, a duotone discrepancy, an operator changing a filter-module gain mid-record — all land on
an existing class with no new mechanism.

2. stream is the primitive; fetch is derived

• stream(channels, duration, chunk_s) -> Iterator[Capture], with fetch consuming it.
• Blocks are verified adjacent — block k+1 begins exactly where k ended — and concatenated.
• The request stays open for the whole record. Per spec §4.3.1, many channels are not recorded
  to disk and must be captured live or they are unretrievable; N back-to-back getdata calls
  would lose samples at every boundary. This is a correctness requirement, not a style choice.

Why chunk at all — strongest reason first

1. Fault detection during the record. A single blocking fetch discovers a framebuilder reboot, a
   lost test point or a stream underrun hours late — spec §4.2's campaign estimates run to 2.5 h —
   and leaves the #16 SIGINT handler nowhere to land.
2. Bounded framebuilder resource use. The binding constraint is the resources available to the
   framebuilder: hardware with less memory, and/or a machine also running other tasks so memory
   is shared. Either, or both.
3. A weak empirical driver, recorded as such. Framebuilders failing to provide long data stretches
   has actually been observed — the one #32 item that is not hypothetical. The cause was never
   investigated (candidates: a test-point timeout, framebuilder resource limits, a client-side
   limitation) and workarounds are commonly used instead. Not authoritative: no numbers, no logs,
   no root cause. It supports the two architectural reasons; it is not their basis.

Concatenation is not stitching

operation	rule
concatenation of verified-adjacent blocks	required — this is the continuous record, and it is invisible downstream
fabrication — zero-fill, interpolate, resample or realign across a hole	forbidden always: synthetic samples in X or Y is §2.1's failure class
gap-tolerant reassembly — analysing two contiguous runs separated by a known GPS gap	disallowed

This does not weaken §2.1. That section forbids correcting the buffer; adjacency checking only
verifies it. Gap-tolerant reassembly is recorded as considered-and-rejected in spec §4.3.2 so it is
not re-derived: a common phase shift does cancel in the ratio-of-averages, but the period straddling
the seam carries a discontinuity that appears differently in X (a raw drive jump) than in Y (that jump
filtered by the plant), so Y_seam ≠ H·X_seam. Disallowed as policy.

3. Retry policy — asymmetric on two axes

• Passive reads retry — bounded, with backoff, for TransportUnavailable / TestpointTimeout.
  A passive record may also be re-attempted after a gap: it carries no excitation.
• Excited records never retry automatically. Re-taking one means re-injecting, which is a new
  actuation needing a new approval token under Rule 2 (#17).
• A retry never silently extends a record's duration or the campaign's time budget.

4. What TwinTransport must not pretend (#12)

It must raise every type and emit chunked blocks so all of the above is testable off-hardware. But
respect the real hardware/simulation difference rather than papering over it: in the twin, GPS is
instantiated at runtime and each mdl.run() is a fresh evolution, so continuity across records
is meaningless there and re-acquiring a whole stretch is cheap. Neither holds on hardware, where GPS
comes from an antenna (best case) or an NTP server (worst case). A gap within one twin record is
representable — keep running cycles, do not fetch them — and that is what the gap tests should use.

Note also that an NTP-disciplined clock can step backwards, so GPS monotonicity stays a check and
a backwards step is a TimingFault.

Config surface

Minimal, honouring #31's "no new configuration" instinct: only cds.read_chunk_s (validated
<= segment_duration) and cds.passive_read_retries, both with in-code defaults. The chunk-size
default is provisional — per the feasibility gate, no value can be justified without a framebuilder
measurement, and the number belongs to that machine's available resources, not to us.

---

Campaign: CDS hardware backend · branch feat/cds-hardware-backend
· spec · plan · handoff
Code is deferred until the plan and issues have been reviewed.