Publish outputs to SciCat, and how an output becomes a file

Abierto
  • ¿Algo no está bien?
#152 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Necesita aclaración
Estado de actividad
Activo
Stack tecnológico
python
Área
backend, data

Línea de trabajo

Read the linked systems and tensions requirements, stories F1 and H3, and issue #151 to understand the publishing requirements and unresolved choices. Inspect the developer README for the client.publish signature and the Store protocol plus testing.FakeStore to see the current storage boundary. The work is done when both callers can publish derived datasets with provenance and file links, and the open format, storage, cleanup, and identity questions are resolved.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Put an output in SciCat as a derived dataset, with its provenance as metadata and a link to a written file.

Required by: the Goal ("upload them to SciCat"), "batch reduction must put its results into SciCat, or at least let users do so" (systems), and stories F1 and H3. Only results in SciCat are found later, and the first release has no connect(), so SciCat is also how users find the results of automatic reduction (#151).

Two callers
  • The batch application, when a person chooses, likely with scicat_widget for the UI.
  • The automatic-reduction service, with no frontend open.
Proposal

A library function that both call, which reads the output and its provenance through the client:

pid = publish(client, record.ref('iofq'), scicat)

So the backend gains no publisher, and client.publish (signature only, in the developer README) goes away.

How an output becomes a file

SciCat holds metadata and file paths, not the files, so a published output needs a file in a known format at a path. The Store protocol has only write and read by record ID and output name, and leaves the format and the layout to the store. Its only implementation is testing.FakeStore.

To decide:

  • the format per output type;
  • the layout on disk, and where a file must be for SciCat to link to it;
  • whether publishing reads the store's file or writes a file of its own;
  • how files are dropped by proposal, label, or age (H1), given that a published dataset stays after the history is dropped.

No wire format is needed for the first release. A later transport may share the format.

Open requirement questions
  • May the service create datasets for a proposal, and who may delete them (systems, Open)?
  • Do results reach SciCat automatically, or only when a person chooses (tensions, Open)?
  • How does a user's SciCat token reach the batch application, and which identity does the service use?
  • Software versions in the provenance (provenance.software, F2).
Lenguaje dominante
Python
Estrellas
0
Forks
1
Merge medio
15 h 40 min
PR fusionados (30 d)
59

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de scipp/essapps

Todos los issues de scipp/essapps

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.