Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

Design box health tests for manufacturing QA and firmware releases

Ouverte
  • Quelque chose ne va pas ?
#339 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
8/100
Type d'issue
Fonctionnalité
Clarté
Plutôt claire
Activité
Active
Stack technique
lua, shell
Domaine
testing-qa

Piste de recherche

This is a design proposal, not an implementation task: it asks for agreement on hardware profiles, the report contract and the verdict model. The payload names src/services/ui/diagnostics.lua, src/services/ui/http/request.lua (the /api/diagnostics stub) and the grand-pipeline runner-scripts/image_installer/execute_diagnostics.sh. Done means the review questions are answered and split into focused implementation issues; the first concrete step is making those consumers fail closed on stubs and count mismatches.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

enhancement

Goal

We need a repeatable way to establish that a newly assembled BigBox or GetBox works as a normal Wi-Fi box, qualify a firmware release on known-good hardware, and assess health after an update. These three workflows should share diagnostic checks and evidence, while differing in physical steps and the confidence they can claim.

Hardware acceptance must expose faults in required components, slots, ports, radios, power features, and client paths. Release qualification must test the same functional contract plus updates, persistence, recovery, and longer operation. An unattended post-update check should report what returned and what remains untested; without an external client probe, it cannot certify Wi-Fi client forwarding.

Why the old diagnostics are insufficient

The v0.10.7 diagnostics largely counted installed packages, running processes/services, modem entries, provisioned files, and basic cloud requests. Some checks prove less than their names suggest: modem enumeration returns an empty missing list even when its count is wrong, and the old UniFi check only verifies that an IP is configured.

The current Lua /api/diagnostics handler returns HTTP 200 with stub: true, zero expected checks, and empty failure lists. The BigBox diagnostic runner decides success solely from nonempty missing lists. Running that function against local fixtures returned “All diagnostic checks passed” for the stub, an expected-two/observed-one modem mismatch, {}, and malformed JSON. An explicit missing-package entry did fail. This path must cease being used as an acceptance gate until its consumers fail closed.

Boot-slot update success is also a different claim from functional health. A box can boot the intended image while a required component or client path remains broken.

Proposed testing model

Define a versioned expected hardware profile for each supported model and revision, independently of what the box discovers. Pair it with the exact release/image manifest. A missing required component must not cause the test suite to silently shrink. GetBox’s current default image runs legacy devicecode; the first design should include a legacy adapter, not assume every product has migrated to Lua.

Use one catalogue of checks with three profiles:

Profile Required evidence
Hardware acceptance Every required component and connector; each modem/SIM path; each required Wi-Fi band and client segment; wired and cellular backhauls; failover; fitted power/I/O responses; provisioning, persistence, reboot, and cloud operation
Release qualification Acceptance on known-good units, supported upgrade paths, host and component image identity, configuration persistence, recovery, shaping/policy detail, and soak
Post-update health Intended image and components running, service/config readiness, persistent state, and bounded probes over paths that were expected to work before update; explicitly record untested physical and client behavior

Test the client experience from an isolated external client: association on the intended radio, DHCP, DNS, forwarding, and a fresh response through the selected backhaul. Test each path separately; an aggregate “online” state can conceal a failed modem. SIM or cable removal remains an operator step, but the runner should verify the loss and recovery automatically. Use controlled, sustained client traffic to test shaping; the current short box-originated WAN speed estimate is not a shaping qualification test.

Every check should record its expected predicate, prerequisites, evidence source and acquisition time, path identity, bounded wait, and result. Results are PASS, FAIL, WARN, BLOCKED, N/A, or NOT_TESTED. An incomplete, stale, unsupported, partial, malformed, or stubbed result cannot become a pass. The overall verdict is FAIL for a failed required check, INCOMPLETE for unresolved required checks, and PASS only when the required suite is complete. Installation outcome and health verdict remain separate.

Start with a guided station runner and machine-readable report, reusing relevant capabilities from the existing QA tool. Keep a small independent on-box collector for post-update checks so failure of Lua or the local UI cannot erase the result. Share probes and report evidence with future field troubleshooting, while keeping factory and release acceptance criteria separate.

Proposed order of work

  1. Make existing diagnostic consumers reject stubs, bad HTTP/JSON, missing sections, and count mismatches. Show “unavailable/incomplete” in the UI instead of a health percentage.
  2. Agree the hardware profiles, connector mappings, required features, release provenance, station setup, and report contract. Pilot bigbox-v1-cm-2 and GetBox V1; add the other supported BigBox profiles before claiming complete product coverage.
  3. Build and validate the guided station workflow with known-good SIMs, backhaul, cables, an isolated client adapter, and a local probe server. Calibrate performance and recovery limits on representative good units.
  4. Add unattended post-update collection, correlation across reboot, deferred report delivery, and a release matrix covering supported upgrade and recovery paths. Retain existing boot/rollback policy until separately reviewed.

Review requested

  • Is the shared functional definition of a working box sufficient for hardware acceptance and release qualification?
  • Which model/revision and fitted-feature profiles should be in the first pilot?
  • Which checks should be required when a carrier or cloud dependency is unavailable? The proposal treats an unverified required check as incomplete rather than a hardware failure or a pass.
  • What equipment and acceptable per-box duration can the hardware team support?
  • What client canary, if any, can provide forwarding evidence after unattended updates?

Related: #329 tracks release functional differences; #306 concerns speed tests. This issue is for the acceptance design and its evidence contract, with implementation work to be split into focused issues after review.

Langage dominant
Lua
Étoiles
1
Forks
0
Merge moyen
5 j 15 h
PR mergées (30 j)
3

Préparer son environnement

Ouvrir dans Codespaces

Lance le conteneur de développement du projet dans votre navigateur, avec votre propre compte GitHub.

  • Aucun Dockerfile ni fichier Docker Compose
  • Aucun modèle de pull request
  • Aucun guide de contribution

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de jangala-dev/devicecode-lua

Toutes les issues de jangala-dev/devicecode-lua

Issues similaires

Plus d'issues Lua

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.