LoKI: handle all detector banks in the specs and bindings

Abierto
  • ¿Algo no está bien?
#166 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
python
Área
backend-api-design, data

Línea de trabajo

Read the design and blocked-by sections in this issue, then follow ess/sans/workflow.py and the CALIBRATED_DETECTOR, MASKS, CREATE_MASK, BEAM_CENTRE, and IOFQ specs and bindings in ess.instruments.loki. The work depends on the interface chosen by esssans for processing multiple banks; start by checking the resolution of scipp/ess#763 and scipp/ess#834. Done means all nine banks are handled with masks validated per bank and plots and output views represent bank data correctly.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

The specs and bindings in ess.instruments.loki handle one detector bank, because they are bound to LokiAtLarmorWorkflow. LoKI has 9 banks. This issue records the plan for handling all of them. Part of #24.

Design

There is one CALIBRATED_DETECTOR record per run, and the banks are inside its value. So a request has the same shape for 1 or 9 banks:

class DetectorImage(BaseModel, arbitrary_types_allowed=True):
    banks: dict[str, sc.DataArray]   # banks differ in shape, so they can't be one array
    sample_position: sc.Variable
    source_position: sc.Variable

class PixelMasks(BaseModel, arbitrary_types_allowed=True):
    masks: dict[str, dict[str, sc.Variable]]   # bank -> mask name -> bool over any of the bank's dims

detector = client.submit(CALIBRATED_DETECTOR, params={'run': file(run=60339)})
masks = client.submit(MASKS, params={'detector': detector.ref('detector'),
                                     'mask_files': [file(path=mask_file)]})
centre = client.submit(BEAM_CENTRE, params={'detector': detector.ref('detector'),
                                            'masks': masks.ref('masks')})
iofq = client.submit(IOFQ, params={'banks': ['loki_detector_0', ...], ...})
Spec Params Binding
CALIBRATED_DETECTOR unchanged loop over banks, keep only bins.sum(), so events of one bank are in memory at a time
MASKS unchanged read each file by bank, see below
CREATE_MASK unchanged _angle_to_beam per bank; an angle range applies to all banks
BEAM_CENTRE unchanged uses one fixed bank, loki_detector_0, since banks 1-8 do not contain the beam axis
IOFQ adds banks: list[str] masks picked per bank, see "Blocked by"

Per-bank I(Q), as asked for in docs/requirements/sans.md, is a batch of IOFQ records with banks=[name], all referencing the same masks and centre.

Plot(inputs=...), Listing.fitting, describe, and references between records need no change. A nested per-bank value of this shape went through an in-process client unchanged.

Mask files

One file is one mask, named by its file, as now. It maps the name of a bank to that bank's part of the mask. A tube index only means something within its bank, so the bank has to be in the file. The file format and the naming inside a file are under discussion in scipp/ess#834, and this plan depends only on the following:

  • A mask has only the dims it needs, such as a tube. It is not always per pixel.
  • A bank not listed has no mask from that file. A beam-stop mask only concerns the rear bank.
  • A bank name the detector does not have is refused, and so is a mask that does not fit its bank's dims and sizes. Both fail in MASKS, not during a long I(Q). This is why MASKS keeps its detector reference.
  • The XML files of pixel IDs stay readable if we still need them: their reader builds the same mapping from each bank's detector_number. Detector numbers do not overlap between banks in loki-coda-5-pulses.hdf.
def compute_masks(params: MasksParams) -> MasksOutputs:
    masks = {bank: {} for bank in params.detector.banks}
    for path in params.mask_files:
        for bank, mask in _read_mask(path, params.detector).items():  # one reader per format
            if bank not in masks:
                raise ValueError(f'{path.name}: unknown bank {bank}')
            params.detector.banks[bank].assign_masks(check=mask)  # raises if dims don't fit
            masks[bank][path.name] = mask
    return MasksOutputs(masks=PixelMasks(masks=masks))

Plots

on_detector makes one x-y histogram, which mixes banks at different distances. detector_image and masks_on_image need one figure per bank or a mosaic. centre_on_image should draw only the bank the centre came from, since the cross is valid in the plane at centre.z.

The generic view in panel/outputs.py needs a change too. Outputs._plot draws only arrays at the top level of an output, so it would show nothing for banks, where it now shows counts with sliders. _brief would print a raw repr of the dict.

Blocked by

ess.loki processes one bank at a time. ess/sans/workflow.py on main says so, because esssans has no way to merge the I(Q) of banks. Merging before normalization is step 1 of the plan in scipp/ess#763. esssans is meant to handle several banks by default.

  • IOFQ is blocked: how _iofq_pipeline selects banks and gives each its masks depends on the interface esssans ends up with.
  • CALIBRATED_DETECTOR, MASKS, CREATE_MASK, and BEAM_CENTRE could loop over NeXusDetectorName today, but we would be guessing the same interface. The only full reductions we test on have one bank (Larmor). loki-coda-5-pulses.hdf has 9 banks, but gives NaN for the centre of mass.

So this waits for esssans.

Discarded alternatives

  • One record per bank. The file is read 9 times, and each downstream request needs lists of references (detectors: [ref × 9], masks: [ref × 9]), which the form shows as 9 pickers each. Nothing stops pairing the masks of one bank with the detector of another. This is also against ADR 0006: structure below a row belongs to the driver.
  • detector: dict[str, DetectorImage] with keyed references. Listing.fitting offers only whole outputs and read_inputs refuses keyed references, and the keys of the detector and the masks would have to agree.
  • A file per bank, with the bank in the params or the file's metadata. One mask becomes up to 9 files, or the per-bank lists come back into every request.

Open questions

  • The file format, and whether a bank can hold several arrays in one file, for example one by tube and one by straw.
  • Mask dim names are those of DETECTOR_BANK_SIZES. A file written with other names is refused.
  • BEAM_CENTRE and each plot read all banks to use one: about 140 MB for 3.2 million pixels, assuming float64 counts and variances, not measured.
  • Do we still need the XML pixel-ID files for LoKI at ESS?
Lenguaje dominante
Python
Estrellas
0
Forks
1
Merge medio
9 h 23 min
PR fusionados (30 d)
65

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de scipp/essapps

Todos los issues de scipp/essapps

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.