bug(nodes): duplicate ObservedNode rows from a non-atomic get-then-create

Aperta
  • Qualcosa non va?
#480 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
30/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
backend, databases

Direzione di ricerca

Start by reading packets/services/base.py and packets/receivers.py, where the issue identifies the non-atomic get-then-create. Then inspect the models and migrations to map the Meshtastic node number, unique constraint, and foreign keys that the dedupe must preserve; review the 2.2.0 migration for its fail-closed and dry-run approach. Done means preventing concurrent duplicates, safely merging existing rows before adding the partial unique index, and naming the node in the remaining error response.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug tech-debt

Problem

Some Meshtastic node numbers have two or more ObservedNode rows. In pre-prod there are 38 duplicate pairs, and the prod identity backfill on 2026-11-04 reported 8 duplicate rows.

Cause (confirmed by a verifier): the node is created with a non-atomic get-then-create in packets/services/base.py and packets/receivers.py. Two packets from a new node, ingested within milliseconds of each other (0–8 ms apart in the pre-prod pairs), both miss the get and both create a row.

Effects:

  • Packet uploads that then look the node up hit MultipleObjectsReturned and return 500. The pre-prod bot log had 6 of these on 2026-11-04, and the 400 they produce doesn't name the node.
  • Ghost rows keep a synthesised Unknown Node !xxxxxxxx name while the real row has the node's name, so search and the UI can show two nodes.
  • NodeDB upserts touch .first(), so it's unpredictable which row they update.

Expected

  1. Prevent it: make the create atomic, with get_or_create backed by a unique constraint and a retry on IntegrityError.
  2. Constraint: add a unique (partial) index on the Meshtastic node number, for example meshtastic_node_id where the protocol is Meshtastic.
  3. Clean up existing rows: a data migration or management command that merges each duplicate set into one row: the one with the most history, or the oldest. It re-points every FK (packets, observations, positions, telemetry, claims, watches, and so on), keeps the best name, and deletes the extras. It needs a --dry-run, and should run before the unique index is added.
  4. Error message: the remaining 500/400 should name the node.

Notes

  • This predates the Meshtastic 2.8 work. The rows date from before 2026-11.
  • The dedupe has to run in prod, so it needs the same care as the 2.2.0 migration: fail closed, dry-run first.
  • Related: #436 (ingest correctness review), #437 (2.8: identity linking will need one row per node number).
Lingua principale
Python
Stelle
1
Fork
0
Merge medio
23h 9m
PR unite (30g)
15

Preparare l'ambiente

  • Include un Dockerfile o un file Docker Compose
  • Ha un modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di pskillen/meshflow-api

Tutte le issue di pskillen/meshflow-api

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.