ENH: Remove the filter index -1 report of a failed final pipeline snapshot

Abierto
#1,948 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 6 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Refactorización
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
cpp, python
Área
api, backend

Línea de trabajo

Start by reading Pipeline::executeFrom() in src/simplnx/Pipeline/Pipeline.cpp and list every reader of AbstractPipelineNode::getDataStructure() on a Pipeline. Trace how the final snapshot is used for local execution, nested pipelines, disabled filters, early stops, and empty pipelines. Done means removing the copy and keeping those readers correct, with the requested C++ and Python coverage and index -1 handling removed where no longer needed.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement

Summary

Remove the "filter index -1" report for a failed final pipeline snapshot. The preferred change is to remove the final snapshot copy, so that this failure cannot occur. This is a follow-up to the out-of-core review PR tree. It starts at #1933 ("BUG: Report store flush, pipeline completion and recovery failures"), and that PR keeps index -1 for now.

How it works in #1933

At the end of a run, Pipeline::executeFrom() copies the DataStructure into the pipeline node with setDataStructure(). This copy is the final pipeline snapshot (Pipeline.cpp#L376-L396). If the copy throws, the errors are -272 (memory allocation) or -6071. No filter owns this failure, and the pipeline node has no place to keep errors. Also, no pipeline-level signal carries error text: PipelineFaultSignal sends only a FaultState. So the errors are sent with the per-filter signal and a filter index that is not real:

sendFilterFaultDetailMessage(-1, {}, snapshotErrors);

Each listener must know that -1 means "the pipeline itself":

  • nxrunner prints the errors as "Pipeline completion errors" (CliObserver.cpp#L25).
  • DREAM3D-NX (the companion PR tree) shows index -1 in the Issues list as "Pipeline: name".
  • The Python Pipeline.execute() binding does not handle it. ExecutePipeline() builds its result from the stored fault state of each filter and sub-pipeline node (simplnxpy.cpp#L522-L573). The snapshot errors are only sent, not stored, so a failure of only the final snapshot gives an invalid result with an empty error list. This was found by reading the code. No test was run.

On develop, the same copy has no try. The exception leaves Pipeline::execute(), and nxrunner and DREAM3D-NX catch it and report a failure. This failure is not a persistence or flush failure. It occurs only when the copy runs out of memory or a DataObject copy throws. No test covers it.

Why it is needed

Index -1 gives an existing signal a second meaning. Every current and future listener must handle it, and one listener (Python) already does not. The design was questioned during review of #1933.

Requested change (preferred: remove the copy)

The final snapshot copies the same DataStructure that the last executed filter node copied a moment before, in AbstractPipelineNode::endExecutionChecked() (AbstractPipelineNode.cpp#L137). If the pipeline's result is the snapshot of its last executed node, or a shared reference to it, then the final copy, its failure path and index -1 can all be removed. DREAM3D-NX already reads the last filter's snapshot, not the pipeline's. Python reads the DataStructure that it passes in.

These readers depend on the pipeline's own snapshot today, and the change must keep them correct:

  • Pipeline::execute(const std::atomic_bool&) runs on a local DataStructure, so its result exists only in the pipeline snapshot (Pipeline.cpp#L193-L197).
  • A pipeline nested in another pipeline: executeFrom(index) reads at(index - 1)->getDataStructure(), and that node can be a sub-pipeline (Pipeline.cpp#L416).
  • Disabled filters at the end of the pipeline, a run that stops early (an error or a cancel), and an empty pipeline.

All readers of AbstractPipelineNode::getDataStructure() on a Pipeline must be listed before the change.

Fallback: pipeline-level errors

If removing the copy cannot keep those readers correct, give the pipeline its own errors:

  • The pipeline stores its completion errors and warnings.
  • A pipeline-level signal sends them, for example a new fault-detail signal without a filter index, or errors added to PipelineFaultSignal.
  • nxrunner, DREAM3D-NX and Python read them from the pipeline, with no special index.

The signal design for the fallback is still open.

Tests

  • C++: a test where only the final pipeline step fails (for example, a DataObject whose copy throws), if that failure path remains. It checks the result of Pipeline::execute(), the stored errors and the signals.
  • Python: one test of Pipeline.execute() for the same case, which checks that the result contains the error, or that the case can no longer occur.
  • Remove the index -1 handling from nxrunner CliObserver, DREAM3D-NX and their tests, if the change makes it unnecessary.
Lenguaje dominante
C++
Estrellas
18
Forks
15
Merge medio
4 d 23 h
PR fusionados (30 d)
42

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de BlueQuartzSoftware/simplnx

Todos los issues de BlueQuartzSoftware/simplnx

Issues similares

Más issues de C++

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.