Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Design box health tests for manufacturing QA and firmware releases

Abierto
  • ¿Algo no está bien?
#339 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
8/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
lua, shell
Área
testing-qa

Línea de trabajo

This is a design proposal, not an implementation task: it asks for agreement on hardware profiles, the report contract and the verdict model. The payload names src/services/ui/diagnostics.lua, src/services/ui/http/request.lua (the /api/diagnostics stub) and the grand-pipeline runner-scripts/image_installer/execute_diagnostics.sh. Done means the review questions are answered and split into focused implementation issues; the first concrete step is making those consumers fail closed on stubs and count mismatches.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement

Goal

We need a repeatable way to establish that a newly assembled BigBox or GetBox works as a normal Wi-Fi box, qualify a firmware release on known-good hardware, and assess health after an update. These three workflows should share diagnostic checks and evidence, while differing in physical steps and the confidence they can claim.

Hardware acceptance must expose faults in required components, slots, ports, radios, power features, and client paths. Release qualification must test the same functional contract plus updates, persistence, recovery, and longer operation. An unattended post-update check should report what returned and what remains untested; without an external client probe, it cannot certify Wi-Fi client forwarding.

Why the old diagnostics are insufficient

The v0.10.7 diagnostics largely counted installed packages, running processes/services, modem entries, provisioned files, and basic cloud requests. Some checks prove less than their names suggest: modem enumeration returns an empty missing list even when its count is wrong, and the old UniFi check only verifies that an IP is configured.

The current Lua /api/diagnostics handler returns HTTP 200 with stub: true, zero expected checks, and empty failure lists. The BigBox diagnostic runner decides success solely from nonempty missing lists. Running that function against local fixtures returned “All diagnostic checks passed” for the stub, an expected-two/observed-one modem mismatch, {}, and malformed JSON. An explicit missing-package entry did fail. This path must cease being used as an acceptance gate until its consumers fail closed.

Boot-slot update success is also a different claim from functional health. A box can boot the intended image while a required component or client path remains broken.

Proposed testing model

Define a versioned expected hardware profile for each supported model and revision, independently of what the box discovers. Pair it with the exact release/image manifest. A missing required component must not cause the test suite to silently shrink. GetBox’s current default image runs legacy devicecode; the first design should include a legacy adapter, not assume every product has migrated to Lua.

Use one catalogue of checks with three profiles:

Profile Required evidence
Hardware acceptance Every required component and connector; each modem/SIM path; each required Wi-Fi band and client segment; wired and cellular backhauls; failover; fitted power/I/O responses; provisioning, persistence, reboot, and cloud operation
Release qualification Acceptance on known-good units, supported upgrade paths, host and component image identity, configuration persistence, recovery, shaping/policy detail, and soak
Post-update health Intended image and components running, service/config readiness, persistent state, and bounded probes over paths that were expected to work before update; explicitly record untested physical and client behavior

Test the client experience from an isolated external client: association on the intended radio, DHCP, DNS, forwarding, and a fresh response through the selected backhaul. Test each path separately; an aggregate “online” state can conceal a failed modem. SIM or cable removal remains an operator step, but the runner should verify the loss and recovery automatically. Use controlled, sustained client traffic to test shaping; the current short box-originated WAN speed estimate is not a shaping qualification test.

Every check should record its expected predicate, prerequisites, evidence source and acquisition time, path identity, bounded wait, and result. Results are PASS, FAIL, WARN, BLOCKED, N/A, or NOT_TESTED. An incomplete, stale, unsupported, partial, malformed, or stubbed result cannot become a pass. The overall verdict is FAIL for a failed required check, INCOMPLETE for unresolved required checks, and PASS only when the required suite is complete. Installation outcome and health verdict remain separate.

Start with a guided station runner and machine-readable report, reusing relevant capabilities from the existing QA tool. Keep a small independent on-box collector for post-update checks so failure of Lua or the local UI cannot erase the result. Share probes and report evidence with future field troubleshooting, while keeping factory and release acceptance criteria separate.

Proposed order of work

  1. Make existing diagnostic consumers reject stubs, bad HTTP/JSON, missing sections, and count mismatches. Show “unavailable/incomplete” in the UI instead of a health percentage.
  2. Agree the hardware profiles, connector mappings, required features, release provenance, station setup, and report contract. Pilot bigbox-v1-cm-2 and GetBox V1; add the other supported BigBox profiles before claiming complete product coverage.
  3. Build and validate the guided station workflow with known-good SIMs, backhaul, cables, an isolated client adapter, and a local probe server. Calibrate performance and recovery limits on representative good units.
  4. Add unattended post-update collection, correlation across reboot, deferred report delivery, and a release matrix covering supported upgrade and recovery paths. Retain existing boot/rollback policy until separately reviewed.

Review requested

  • Is the shared functional definition of a working box sufficient for hardware acceptance and release qualification?
  • Which model/revision and fitted-feature profiles should be in the first pilot?
  • Which checks should be required when a carrier or cloud dependency is unavailable? The proposal treats an unverified required check as incomplete rather than a hardware failure or a pass.
  • What equipment and acceptable per-box duration can the hardware team support?
  • What client canary, if any, can provide forwarding evidence after unattended updates?

Related: #329 tracks release functional differences; #306 concerns speed tests. This issue is for the acceptance design and its evidence contract, with implementation work to be split into focused issues after review.

Lenguaje dominante
Lua
Estrellas
1
Forks
0
Merge medio
5 d 15 h
PR fusionados (30 d)
3

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

  • Sin Dockerfile ni archivo de Docker Compose
  • Sin plantilla de pull request
  • Sin guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de jangala-dev/devicecode-lua

Todos los issues de jangala-dev/devicecode-lua

Issues similares

Más issues de Lua

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.