Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

dandi upload: STATUS/MESSAGE for Zarr assets is uninformative

Abierto
  • ¿Algo no está bien?
#1,893 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
52/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Tranquilo
Stack tecnológico
python
Área
cli

Línea de trabajo

Comienza con check_replace_asset() en dandi/upload.py y ZarrAsset.iter_upload() en dandi/files/zarr.py, comparando el comportamiento de omisión existente para activos que no son Zarr con los datos de diff de Zarr. Se considera terminado cuando los activos Zarr sin cambios informan skipped/file exists, mientras que los activos modificados exponen el mensaje y el tamaño correspondientes de carga, modificación o eliminación, incluido el comportamiento de parche de --zarr-mode.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement UX zarr

claude produced out of experience in

Summary

dandi upload reports uninformative and slightly misleading STATUS /
MESSAGE values for Zarr assets. Raised by @kabilar during review of
#1816 (see comment).

Observed behavior

1. Unchanged Zarr is reported as done / exists - reuploading

A dandiset whose local Zarr is bit-identical to the remote Zarr renders
as:

PATH                                              SIZE      ...  STATUS   MESSAGE
...atives/dandi-cli-1816-test/HG9_Z1_Y49.nii.zarr 5.9 GB    ...  done     exists - reuploading

...where the same-hash short-circuit for a regular blob would render
skipped / file exists.

2. Modified Zarr uses the same non-descriptive message

After a local edit, the same asset renders done / exists - reuploading
with no indication of what changed — no distinction between additions,
deletions, and modifications, and no idea of the affected size (which
matters because the SIZE column reflects the whole-Zarr size, not the
delta).

Root cause

check_replace_asset() in dandi/upload.py:547-548 unconditionally
short-circuits Zarr assets:

if isinstance(local_asset, ZarrAsset):
    return (True, {"message": "exists - reuploading"})

It never compares local vs remote, and never differentiates additions
from modifications from deletions. In contrast, the non-Zarr path at
dandi/upload.py:579-583 does perform an etag/mtime comparison and
returns skip_file("file exists") (i.e. STATUS=skipped) when the
local and remote copies match.

Proposed changes

Change 1 -- Detect unchanged Zarr and skip

For a ZarrAsset where the local tree matches the remote Zarr (e.g. by
comparing per-entry digests, or by comparing the aggregated Zarr
checksum), return skip_file("file exists") so that STATUS=skipped / MESSAGE=file exists, matching the non-Zarr case.

Note that the diff needed to decide "unchanged" is currently only
computed inside ZarrAsset.iter_upload() (dandi/files/zarr.py) after
Zarr registration. Two implementation choices:

(a) Do a cheap upfront comparison in check_replace_asset (e.g. via
per-entry digest listing) at the cost of an extra API round-trip
per asset.
(b) Let iter_upload yield an early skipped status once it has
computed the diff, and have the caller replace the initial
exists - reuploading marker. This avoids the upfront cost
but leaks knowledge about Zarr internals into the reporter.

Change 2 -- Describe what is changing

When a Zarr is changing, replace exists - reuploading with the
kind and size of the modification. @kabilar's proposed vocabulary:

  • file exists - uploading additional objects (N GB)
  • file exists - deleting objects (N GB)
  • file exists - modifying objects (N GB)

The relevant sizes are already tracked inside
ZarrAsset.iter_upload() via to_upload.total_size and the
per-entry sizes of to_delete
(dandi/files/zarr.py:655,728); they just aren't surfaced as
MESSAGE. This overlaps with implementation choice (b) above.

Interaction with --zarr-mode patch (from #1816)

Once #1816 lands, patch mode does not delete remote-only entries, so
"deleting objects" should not appear for --zarr-mode patch. The
choice of message per mode:

Mode Local == remote Local adds only Local modifies Local also drops entries
full (dflt) skipped - uploading … (N GB) - modifying … (…) - deleting … (…)
patch skipped - uploading … (N GB) - modifying … (…) not applicable

Scope

Orthogonal to #1816 -- misreport exists on master today. Filing
separately so #1816 can land on its current scope and this can be
picked up as a UX follow-up.

Lenguaje dominante
Python
Estrellas
29
Forks
37
Merge medio
14 h 58 min
PR fusionados (30 d)
15

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de dandi/dandi-cli

Todos los issues de dandi/dandi-cli

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.