P5: Anomaly detection

Open
  • Is something wrong?
#20 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
1/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Stale
Tech stack
python, scikit-learn
Domain
data, machine-learning

Research direction

The issue says the implementation is merged and archived; the remaining full-season tuning is tracked in issue #13. Start by reading that follow-up and the linked OpenSpec change and ADR-0010 amendment. Check the backtest on full 2026 logs against known failures and the false-positive rate; completion means the follow-up's tuning is resolved.

Written by the indexing model from the issue text.

Description

roadmap

Size: L · Depends on: IgniteRobotics/flashpoint#17, IgniteRobotics/flashpoint-private#69, IgniteRobotics/flashpoint-private#71, IgniteRobotics/flashpoint-private#74 · OpenSpec: 2026-10-12-p5-anomaly-detection

Status

Update 2026-10-19: merged (IgniteRobotics/flashpoint-private#41) and archived (IgniteRobotics/flashpoint-private#42) on 2026-10-12. What's left is [HUMAN] 9.3, tuning on the full 2026 season, tracked in IgniteRobotics/flashpoint#13. The status below is the text from when it was implemented.

Status: implemented 2026-10-11 (change p5-anomaly-detection, PR into rewrite, stacked on the can-health archive PR IgniteRobotics/flashpoint-private#40). Field check and backtest on the full 2026 logs are open ([HUMAN] 9.3, 10.4).

  • What shipped:
    • flashpoint detect (also the last step of every derive; skipped when nothing changed) writes gold/anomalies/ with SPEC limits, DRIFT and STEP against each unit's own median/MAD baseline, and SIBLING comparison within the robot config's [[siblings]] groups. MODEL (isolation forest) is optional and off by default (ml extra).
    • Every anomaly carries actual vs expected, confidence, a per-match evidence series with its expected band, a "why", causes from config/anomaly-causes.toml, and "seen this before".
    • The Anomalies view (queue, sensitivity, evidence, triage), a navigation badge, Replay badges (a snapshot in static exports), and anomaly-aware History health.
    • Triage is stored append-only in the ledger: local only, backed up, and kept across rebuilds (ADR-0015).
    • flashpoint backtest scores config/failure-labels.toml (seeded with Johnson Q105 and GACMP Q7; battery-proxy removed GACMP Q7, a wiring drop): caught or missed, lead time, and false positives per method per 10 robot-matches.
  • Measured: detect on a 60 × 23 season takes 1.6 s and 222 MB; the backtest on two seasons takes 3.2 s and 256 MB; an uncached Q7 brownout count takes 0.10 s.
  • On real data: the Johnson lake flags the Q105 roller reboot (HIGH) and its 820 ms gap (MED), intake stalls, and brownouts in all four matches. The Q105 extension-follower degradation is missed until more 2026 logs give it a baseline (3 of 5 matches).
  • Deferred:
    • the Slack/Discord webhook, until the backtest shows a tolerable false-positive rate;
    • NOISE (idle-current jitter), which needs a commanded-off feature in gold;
    • STUMPY discords and River online learners;
    • spin-test and current-limit-count metrics (no data yet);
    • a Driver Station log reader (brownouts come from the battery proxy). Still deferred after battery-proxy.
  • Follow-up, change battery-proxy (2026-10-12): the battery proxy is the robot's declared low-current device (the gyro), with a per-20 ms median fallback, so a motor's wiring drop no longer reads as a brownout; the brownout hold is 0.02 s and a logging gap holds a low sample at most 0.1 s. The deferred battery-resistance metric shipped as supply path resistance per motor (mΩ, a baseline metric). See the ADR-0010 amendment.
  • Follow-up, change ui-cross-tab-filters (2026-10-12): the Anomalies queue takes the shared season, robot, event, and unit filters, applied by the server (/api/anomalies; an anomaly passes when one of its flagged matches is in the chosen season and event), and can be grouped by device (a unit by serial, or a robot's battery). The navigation badge stays unfiltered. The anomaly detail names the competitions its evidence comes from, and the evidence chart labels each one with a boundary line between them. Merged as PR IgniteRobotics/flashpoint-private#44 and archived 2026-10-13 (specs synced: view-filters new; anomaly-review, lifetime-trends, match-reports).

Scope

  • Tier 1, rules:
    • temperature thresholds (port 55 °C avg / 65 °C max);
    • stall seconds (high current at near-zero velocity);
    • CAN dropout counts: done from the hoots by CAN health (above), with no DS log reader. Brownout counts still need the DS log reader (LigerBots, 2026);
    • current residual P95 above a threshold.
  • Tier 2, baselines:
    • robust z-score (median/MAD) of each gold feature against that unit's own history. A swapped-in unit starts its own baseline automatically;
    • also compared against the sibling slots (the 4 drive motors should look alike).
  • Tier 3, models:
    • IsolationForest (scikit-learn developers, n.d.) or PyOD detectors (Zhao et al., n.d.) on gold feature vectors;
    • River HalfSpaceTrees for online scoring (River developers, n.d.);
    • STUMPY discords on silver current traces to catch odd shapes (STUMPY developers, n.d.).
  • Output: an anomalies table, badges in the views, and an optional Slack or Discord webhook.
  • Exit:
    • Backtest on the 2025–2026 logs flags at least one known real failure (team pit notes are the ground truth).
    • False-positive rate is low enough that students don't learn to ignore it.

Moved from the rewrite roadmap on 2026-10-19. That doc keeps the dependency graph, migration, quality gates and risks. Status lives here. Author-year citations are in references.md. pitfall N refers to 03-pitfalls-and-performance.md.

Dominant language
Python
Stars
0
Forks
1
Avg merge
18h 37m
Merged PRs (30d)
15

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from IgniteRobotics/flashpoint

All issues in IgniteRobotics/flashpoint

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.