bug(nodes): duplicate ObservedNode rows from a non-atomic get-then-create

Abierto
  • ¿Algo no está bien?
#480 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
30/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
backend, databases

Línea de trabajo

Start by reading packets/services/base.py and packets/receivers.py, where the issue identifies the non-atomic get-then-create. Then inspect the models and migrations to map the Meshtastic node number, unique constraint, and foreign keys that the dedupe must preserve; review the 2.2.0 migration for its fail-closed and dry-run approach. Done means preventing concurrent duplicates, safely merging existing rows before adding the partial unique index, and naming the node in the remaining error response.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

bug tech-debt

Problem

Some Meshtastic node numbers have two or more ObservedNode rows. In pre-prod there are 38 duplicate pairs, and the prod identity backfill on 2026-11-04 reported 8 duplicate rows.

Cause (confirmed by a verifier): the node is created with a non-atomic get-then-create in packets/services/base.py and packets/receivers.py. Two packets from a new node, ingested within milliseconds of each other (0–8 ms apart in the pre-prod pairs), both miss the get and both create a row.

Effects:

  • Packet uploads that then look the node up hit MultipleObjectsReturned and return 500. The pre-prod bot log had 6 of these on 2026-11-04, and the 400 they produce doesn't name the node.
  • Ghost rows keep a synthesised Unknown Node !xxxxxxxx name while the real row has the node's name, so search and the UI can show two nodes.
  • NodeDB upserts touch .first(), so it's unpredictable which row they update.

Expected

  1. Prevent it: make the create atomic, with get_or_create backed by a unique constraint and a retry on IntegrityError.
  2. Constraint: add a unique (partial) index on the Meshtastic node number, for example meshtastic_node_id where the protocol is Meshtastic.
  3. Clean up existing rows: a data migration or management command that merges each duplicate set into one row: the one with the most history, or the oldest. It re-points every FK (packets, observations, positions, telemetry, claims, watches, and so on), keeps the best name, and deletes the extras. It needs a --dry-run, and should run before the unique index is added.
  4. Error message: the remaining 500/400 should name the node.

Notes

  • This predates the Meshtastic 2.8 work. The rows date from before 2026-11.
  • The dedupe has to run in prod, so it needs the same care as the 2.2.0 migration: fail closed, dry-run first.
  • Related: #436 (ingest correctness review), #437 (2.8: identity linking will need one row per node number).
Lenguaje dominante
Python
Estrellas
1
Forks
0
Merge medio
23 h 9 min
PR fusionados (30 d)
15

Preparar el entorno

  • Incluye un Dockerfile o un archivo de Docker Compose
  • Tiene una plantilla de pull request
  • Sin guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de pskillen/meshflow-api

Todos los issues de pskillen/meshflow-api

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.