[Epic] Meshtastic incoming packet pathway: correctness review

Open
  • Is something wrong?
#436 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
15/100
Issue type
Documentation
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
api, backend

Research direction

This is an epic, not a single fix. Start by reading Meshflow/packets/views.py (PacketIngestView), Meshflow/packets/serializers.py and the docs in docs/features/packet-ingestion/. Pick one unchecked review area, such as the portnum inventory, and produce its part of the gap analysis. Done means a child issue is linked here and the matching field docs are updated. Wait for the 2.8.x work in #437 to land before touching shared files.

Written by the indexing model from the issue text.

Description

enhancement needs-refinement

Summary

Epic. Do a thorough end-to-end review of the Meshtastic incoming packet pathway (bot → POST /api/packets/{node_id}/ingest/ → serializers → MtRawPacket / PacketObservation → signals → downstream apps) to make sure every packet type and field we receive is handled deliberately — stored, normalised, or knowingly ignored — rather than silently dropped or mis-parsed.

The 2.8.x firmware work has been split out to #437. Each checkbox below is a candidate child ticket. Raise child issues as they are scoped and link them here.

Sequencing with #437

  • Shared, do now under #437: the portnum/field inventory, test fixtures, and the protobuf/library bump. This epic reuses them rather than redoing them.
  • Correctness bugs found during the 2.8 work are fixed there, in the code it is already touching.
  • The rest of this review runs after the 2.8 must-haves land. It overlaps the remaining 2.8 work only where the two touch different files.

Review areas

Starting points: Meshflow/packets/views.py (PacketIngestView), Meshflow/packets/serializers.py, Meshflow/packets/services/, Meshflow/packets/receivers.py / signals.py, and the docs in docs/features/packet-ingestion/ (meshtastic.md, PACKET_FIELDS.md, DEDUPLICATION.md, signals.md).

  • Portnum inventory: list every portnum the firmware can emit; for each one, record what the bot forwards, what the API does with it (serializer / model / signal), and what happens to anything unhandled (reject, store raw, or drop). Flag the gaps.
  • Telemetry variants: check every Telemetry oneof (device, environment, air quality, power, health, host, local stats, traffic management) and its fields against the current protobufs. Look for fields we drop or name wrongly.
  • Packet envelope fields: hop_start, hop_limit, relay_node, next_hop, via_mqtt, rx_snr, rx_rssi, rx_time, channel, pki_encrypted, public_key, priority, transport_mechanism and so on. Note which we persist per packet and which per observation.
  • Encrypted / undecodable packets: what happens to them today, and is that what we want?
  • Dedup: re-check the dedup key and window (find_existing_packet, DEDUPLICATION.md) against relays, MQTT-bridged copies and multiple observers.
  • Error handling: validation failures, partial payloads and unknown enums. Make sure a single bad field doesn't cost us the whole packet without leaving a log.
  • Bot side: check the meshflow-bot transform (sanitise, channel inference, raw stripping) for anything lost before it reaches the API. If changes are needed there, raise a linked issue in meshflow-bot.
  • Tests: add fixture packets in docs/packets/ for each portnum and variant, and make sure every serializer has coverage.
  • Docs: update meshtastic.md / PACKET_FIELDS.md so they match reality.

Deliverables

  1. A written gap analysis added to docs/features/packet-ingestion/.
  2. A child issue for each concrete fix.
  3. Updated fixtures, tests and field docs.

Child tickets

None yet. Add them here as they are raised.

Related

  • #437: Meshtastic 2.8.x firmware support
  • #319: retire v1 ingest URLs
  • #83: per-observation forwarded-by
Dominant language
Python
Stars
1
Forks
0
Avg merge
1d 20h
Merged PRs (30d)
5

Getting set up

  • Ships a Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from pskillen/meshflow-api

All issues in pskillen/meshflow-api

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.