LoKI: handle all detector banks in the specs and bindings

Read the design and blocked-by sections in this issue, then follow ess/sans/workflow.py and the CALIBRATED_DETECTOR, MASKS, CREATE_MASK, BEAM_CENTRE, and IOFQ specs and bindings in ess.instruments.loki. The work depends on the interface chosen by esssans for processing multiple banks; start by checking the resolution of scipp/ess#763 and scipp/ess#834. Done means all nine banks are handled with masks validated per bank and plots and output views represent bank data correctly.

The specs and bindings in ess.instruments.loki handle one detector bank, because they are bound to LokiAtLarmorWorkflow. LoKI has 9 banks. This issue records the plan for handling all of them. Part of #24.

Design

There is one CALIBRATED_DETECTOR record per run, and the banks are inside its value. So a request has the same shape for 1 or 9 banks:

class DetectorImage(BaseModel, arbitrary_types_allowed=True):
    banks: dict[str, sc.DataArray]   # banks differ in shape, so they can't be one array
    sample_position: sc.Variable
    source_position: sc.Variable

class PixelMasks(BaseModel, arbitrary_types_allowed=True):
    masks: dict[str, dict[str, sc.Variable]]   # bank -> mask name -> bool over any of the bank's dims

detector = client.submit(CALIBRATED_DETECTOR, params={'run': file(run=60339)})
masks = client.submit(MASKS, params={'detector': detector.ref('detector'),
                                     'mask_files': [file(path=mask_file)]})
centre = client.submit(BEAM_CENTRE, params={'detector': detector.ref('detector'),
                                            'masks': masks.ref('masks')})
iofq = client.submit(IOFQ, params={'banks': ['loki_detector_0', ...], ...})

Per-bank I(Q), as asked for in docs/requirements/sans.md, is a batch of IOFQ records with banks=[name], all referencing the same masks and centre.

Plot(inputs=...), Listing.fitting, describe, and references between records need no change. A nested per-bank value of this shape went through an in-process client unchanged.

Mask files

One file is one mask, named by its file, as now. It maps the name of a bank to that bank's part of the mask. A tube index only means something within its bank, so the bank has to be in the file. The file format and the naming inside a file are under discussion in scipp/ess#834, and this plan depends only on the following:

• A mask has only the dims it needs, such as a tube. It is not always per pixel.
• A bank not listed has no mask from that file. A beam-stop mask only concerns the rear bank.
• A bank name the detector does not have is refused, and so is a mask that does not fit its bank's dims and sizes. Both fail in MASKS, not during a long I(Q). This is why MASKS keeps its detector reference.
• The XML files of pixel IDs stay readable if we still need them: their reader builds the same mapping from each bank's detector_number. Detector numbers do not overlap between banks in loki-coda-5-pulses.hdf.

def compute_masks(params: MasksParams) -> MasksOutputs:
    masks = {bank: {} for bank in params.detector.banks}
    for path in params.mask_files:
        for bank, mask in _read_mask(path, params.detector).items():  # one reader per format
            if bank not in masks:
                raise ValueError(f'{path.name}: unknown bank {bank}')
            params.detector.banks[bank].assign_masks(check=mask)  # raises if dims don't fit
            masks[bank][path.name] = mask
    return MasksOutputs(masks=PixelMasks(masks=masks))

Plots

on_detector makes one x-y histogram, which mixes banks at different distances. detector_image and masks_on_image need one figure per bank or a mosaic. centre_on_image should draw only the bank the centre came from, since the cross is valid in the plane at centre.z.

The generic view in panel/outputs.py needs a change too. Outputs._plot draws only arrays at the top level of an output, so it would show nothing for banks, where it now shows counts with sliders. _brief would print a raw repr of the dict.

Blocked by

ess.loki processes one bank at a time. ess/sans/workflow.py on main says so, because esssans has no way to merge the I(Q) of banks. Merging before normalization is step 1 of the plan in scipp/ess#763. esssans is meant to handle several banks by default.

• IOFQ is blocked: how _iofq_pipeline selects banks and gives each its masks depends on the interface esssans ends up with.
• CALIBRATED_DETECTOR, MASKS, CREATE_MASK, and BEAM_CENTRE could loop over NeXusDetectorName today, but we would be guessing the same interface. The only full reductions we test on have one bank (Larmor). loki-coda-5-pulses.hdf has 9 banks, but gives NaN for the centre of mass.

So this waits for esssans.

Discarded alternatives

• One record per bank. The file is read 9 times, and each downstream request needs lists of references (detectors: [ref × 9], masks: [ref × 9]), which the form shows as 9 pickers each. Nothing stops pairing the masks of one bank with the detector of another. This is also against ADR 0006: structure below a row belongs to the driver.
• detector: dict[str, DetectorImage] with keyed references. Listing.fitting offers only whole outputs and read_inputs refuses keyed references, and the keys of the detector and the masks would have to agree.
• A file per bank, with the bank in the params or the file's metadata. One mask becomes up to 9 files, or the per-bank lists come back into every request.

Open questions

• The file format, and whether a bank can hold several arrays in one file, for example one by tube and one by straw.
• Mask dim names are those of DETECTOR_BANK_SIZES. A file written with other names is refused.
• BEAM_CENTRE and each plot read all banks to use one: about 140 MB for 3.2 million pixels, assuming float64 counts and variances, not measured.
• Do we still need the XML pixel-ID files for LoKI at ESS?