Annotate output is not reproducible: properties[] order varies between runs on identical sources

Abierto
  • ¿Algo no está bien?
#91 1 comentario 0 reacciones 1 asignado Ver en GitHub

@avillar ya está trabajando en esto.

Desde el 8/10/2026.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
56/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
github-actions, python
Área
build-system

Línea de trabajo

Start by tracing how the postprocess builds and serializes properties[] in the outputs under build/annotated/, especially resolvedProperties.json; the issue provides a concrete example with three sibling entries changing order. Reproduce with identical inputs and check whether iteration over an unordered collection determines the output. Done when repeated runs produce byte-stable ordering for these outputs without changing their contents.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Running the postprocess twice over byte-identical sources produces build/annotated/** output that differs only in the order of the properties[] array. The entries are the same; their positions move.

Because this output is committed in our repository, every run looks like a ~1300-file change that contains no change at all, which makes "regenerate and compare" unusable as a check and buries real diffs.

Evidence

Two runs of validate-and-process.yml@v1 (skip-pages: true), producing commits 9496e7c15 and 88eedbaf6 in amds-ldeo/geochemBuildingBlocks. Between those two commits:

  • _sources/ — 0 files changed
  • tapp submodule — unchanged
  • bblocks-config.yaml — unchanged
  • the only non-build/ changes were one GitHub Actions workflow file and a docstring in a tool neither the postprocess nor the schemas read

so the postprocess input was identical. Yet:

  • build/ — 1310 files changed, +166607 −166607 (symmetric)
  • 40 of 40 sampled JSON outputs are order-insensitively identical (comparison canonicalising both dict key order and list element order); 0 genuinely different
  • affected output kinds: resolvedProperties.json (131), index.json (198), _report.json (242), schema-oas3.0.json / .yaml

Minimal example

build/annotated/BaseSchema/tappDefinition/resolvedProperties.json — 215 properties[] entries in both runs, identical multiset, 3 positions differing, a clean rotation:

 [123]  A: schema:actionProcess.schema:step.schema:additionalProperty.schema:unitText
        B: schema:actionProcess.schema:step.schema:additionalProperty.schema:name
 [124]  A: schema:actionProcess.schema:step.schema:additionalProperty.schema:name
        B: schema:actionProcess.schema:step.schema:additionalProperty.schema:propertyID
 [125]  A: schema:actionProcess.schema:step.schema:additionalProperty.schema:propertyID
        B: schema:actionProcess.schema:step.schema:additionalProperty.schema:unitText

Three sibling properties of one object, emitted in a different order each run. That pattern — a small group of siblings rotating — is what an unordered collection (a set, or a dict built in traversal order and iterated without sorting) looks like when it reaches the output.

Suggested fix

Sort properties[] before serialising — by path (and id as a tiebreak) would be stable and also easier to read. More generally, any collection that reaches committed output needs a defined order.

We hit the same class of bug in our own generators and fixed it the same way; we also added a CI check that regenerates under two different PYTHONHASHSEED values and compares, which catches exactly this. If the ordering here derives from set/dict iteration, that check would likely reproduce it in your CI cheaply.

Why it matters downstream

build/annotated/** is committed in our repository because our Pages deploy reads it without regenerating. With non-deterministic ordering:

  • a "regenerate and diff" check cannot distinguish a real change from reordering
  • every source change produces a large spurious diff, so genuine changes are not reviewable
  • the postprocess reports success, so nothing signals that the output is unstable

Happy to supply fuller diffs, the comparison script, or run an instrumented build if that helps isolate it.

Environment

  • opengeospatial/bblocks-postprocess/.github/workflows/validate-and-process.yml@v1, default image_tag
  • observed 2026-10-07
Lenguaje dominante
Python
Estrellas
1
Forks
6
Merge medio
23 h 43 min
PR fusionados (30 d)
7

Preparar el entorno

  • Incluye un Dockerfile o un archivo de Docker Compose
  • Sin plantilla de pull request
  • Sin guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de opengeospatial/bblocks-postprocess

Todos los issues de opengeospatial/bblocks-postprocess

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.