Decide whether components get a quality scale

Aperta
#206 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
github-actions

Direzione di ricerca

Start with the proposed rules in this issue, then read AGENTS.md, the logging guide, and the referenced review changes to understand the existing reliability standards. Decide whether ESPHome should adopt a component quality scale or a single long-uptime marker, and settle the criteria, display location, ownership, and regression policy before implementation.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

Should ESPHome have a quality scale for components, like Home Assistant's integration quality scale but built around our reliability and uptime rules? The core is now at the point where a device can expect years of uptime from a software point of view, but components vary a lot, and nothing tells users or contributors which ones meet that bar.

Where things stand

  • Most, if not all, of the heap churn and fragmentation in core is gone, so long-running devices no longer slowly cut up their heap.
  • Runtime crash reporting is in place, so the crashes that are left show up instead of looking like random reboots.
  • Crashes are way down, and most of the ones still reported come from upstream code (SDKs and libraries), not from ESPHome. They are still ours to fix, because we own the product end to end, but it is a lot better than it was.
  • From a software point of view, most devices can now expect at least 10 years of uptime.
  • Components have not all caught up. Some still allocate on the heap after setup(), and some have other long-uptime problems. Today there is no list of which ones, and no way for a user to tell before they pick one.

Why it might be worth it

  • Contributors: a visible scale with clear steps tends to get people improving components on their own, because they can see what "better" means and claim it.
  • Users: people choosing between two drivers for the same chip could pick the one that is known to be safe for long uptime.
  • Industrial use: that is not a goal of the project, but a clear statement of which components meet a long-uptime bar opens that door without us having to promise anything for the rest.

How Home Assistant does it

Home Assistant grades integrations Bronze, Silver, Gold and Platinum (quality scale). Each tier is a list of concrete rules, and an integration meets every rule of its tier and the tiers below. Each integration keeps a quality_scale.yaml that marks every rule done, todo or exempt (with a reason). Hassfest fails CI when a rule for the claimed tier is neither done nor exempt, reviewers check the rules a script cannot, and the tier is shown on the docs page and in the UI.

Their tiers are about setup flows, translations, discovery and typing. Ours would be about what keeps a device running for years, taken from the rules in AGENTS.md and the logging guide, plus the cleanups reviews keep missing that esphome/esphome#22708 adds to AGENTS.md and the review skill.

A possible shape (starting point only)

  • Bronze, builds and behaves:
    • schema validated, and script/build_language_schema.py extracts it
    • build tests on every platform the component supports
    • dump_config() reports the setup; failures go through mark_failed() with a reason, in setup() only
    • logging follows the logging guide: readings at VERBOSE, no "Setting up" lines, related dump_config() lines in one call
    • a code owner
  • Silver, safe to run for years:
    • no heap allocation after setup() (fixed-size containers, StaticVector, StringRef)
    • scheduler lambdas capture at most 8 bytes; repeating work uses set_interval, never a self-rearming set_timeout
    • no millis() in loop(), and no rate-limit gate shorter than the 16 ms loop
    • runtime read and write failures set status_set_warning() and the next success clears it
    • multi-step command sequences never advance on a malformed answer
    • no state that copies other state (a flag that mirrors a timer, a has_x_ bool beside x_)
  • Gold, lean and proven:
    • constants and string tables kept in flash on ESP8266 (PROGMEM, LOG_STR, PROGMEM_STRING_TABLE); one table where a value is both encoded and decoded
    • members laid out without padding, small codes as uint8_t, flags packed
    • math that depends only on the config done in to_code, and model-specific code behind a cg.add_define()
    • leaf classes final, no dead guards or unused helpers
    • C++ unit tests or integration tests that fail if any path regresses, not only build tests
    • tested on real hardware, with the board and result recorded
  • Platinum (maybe), long-run verified:
    • a soak test with no heap growth and no crash reports over a set period
    • no exemptions

Enforcement ideas

  • A per-component rules file, like Home Assistant's quality_scale.yaml, with the tier claimed in the component's __init__.py; CI checks that every rule for the tier is done or exempt.
  • Some rules can be checked automatically. For example, the host integration tests could wrap malloc and fail when a component allocates after setup().
  • The rest stay with reviewers, as in Home Assistant. The review skill from esphome/esphome#22708 could label each finding with its tier, so an automated review also tells a contributor what the component needs for the next one.

To discuss

  • Do we want a full scale, or only a single "long-uptime ready" marker (roughly Silver above)?
  • Are these the right rules for each tier, and is the order right?
  • Where is the tier shown: the docs page, device builder, or both?
  • Who assigns a tier, and what happens when a component regresses?
Lingua principale
Nessun dato sulla lingua
Stelle
0
Fork
0
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di esphome/backlog

Tutte le issue di esphome/backlog

Issue simili

Altre issue su Backend & API Design

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.