QCFilterTransform emits on every out-of-range record; emit on transitions instead
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 54/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python
- Domain
- data-engineering
Research direction
Start in logger/transforms/qc_filter_transform.py, reading transform() around lines 60–110, then compare the stateful transition handling in GeofenceTransform at geofence_transform.py:258–261. The key design point is the default compatibility behavior, so read server/logger_manager.py:369–389 and the set_active_mode implementations before coding. Done means per-field in/out-of-bounds state emits once on entering a bad state, an optional resume_message fires when all fields recover, and steady bad values no longer repeat the command.
Written by the indexing model from the issue text.
Description
QCFilterTransform returns its message for every record that has a value out of bounds, not when a value goes out of bounds. It keeps no state between calls (logger/transforms/qc_filter_transform.py, transform() lines 60–110): each record is checked against bounds, and if any field fails, the message (or the joined error list) is returned. A sensor that sits at a bad value for an hour therefore produces the message at the sensor's full record rate.
Why it matters
On its own that is noise in whatever the message goes to (a logfile, an EmailWriter, Sealog). But the pattern the v2.7.0 documentation shows for conditional control makes it harmful: a QC logger whose writer is a LoggerManagerWriter, with message set to a logger manager command such as set_active_mode safe. Every implementation of set_active_mode (server/sqlite_server_api.py:441, server/in_memory_server_api.py:165, and the Django and FastAPI equivalents) overwrites every logger's active configuration with the mode's and signals an update.
The logger manager is careful about this: _apply_configs (server/logger_manager.py:369–389) restarts only loggers whose configuration actually changed, and _journal_mode records a mode change only when the mode differs. So a repeated command is harmless to loggers already in the mode's configuration. It is not harmless to a logger an operator has switched by hand in the meantime: that logger's active configuration is different from the mode's, so on the very next out-of-range record it is switched back, restarted, and journaled as config_changed, and that repeats at the sensor's record rate for as long as the value stays out of range. The operator's change cannot stick.
GeofenceTransform already does this the right way: it remembers last_position_inside and emits only when the inside/outside state changes (geofence_transform.py:258–261), with a seconds_between_checks throttle.
Proposal
Make QCFilterTransform emit on transitions, keeping per-field in/out-of-bounds state:
- emit
message(or the error list) when a field first goes out of bounds, and not again while it stays out; - add an optional
resume_messageemitted when all fields are back within bounds, mirroringTimeoutReader/TimeoutWriter(andGeofenceTransform's entering/leaving pair); - optionally a
seconds_between_checksthrottle as inGeofenceTransform.
Compatibility: anyone relying on the per-record behaviour (e.g. counting bad records downstream) could be given an emit_every_record: true flag that preserves the current behaviour, or the transition behaviour could be opt-in via on_transition_only: true for one release. The former seems the better default, since per-record emission is almost never what a logger-manager command wants.
With transition-only emission, a hand-switched logger is reset at most once per excursion, which matches what a genuine mode change does and is what an operator would expect.
🤖 Generated with Claude Code
- Dominant language
- Python
- Stars
- 57
- Forks
- 30
- Avg merge
- 3h 14m
- Merged PRs (30d)
- 56
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from OceanDataTools/openrvdas
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
OceanDataTools/openrvdas#760 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
OceanDataTools/openrvdas#758 · 1 comment ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 25/100
OceanDataTools/openrvdas#748 · 1 comment ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 58/100
OceanDataTools/openrvdas#743 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 35/100
OceanDataTools/openrvdas#721 ·
Maintainers usually reply within 1 day
All issues in OceanDataTools/openrvdas
Similar issues
-
automated endpoint-dead needs-curation
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
almanac-data/climate-almanac#138 ·
Maintainers usually reply within 1 day
-
level/subtask type/enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
wazuh/wazuh-virtual-machines#1091 ·
Maintainers usually reply within 1 day
-
Python: Bug: ChatMessageContent.from_element drops AudioContent and BinaryContent into raw TextContentPossibly taken @halcyon-past claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
microsoft/semantic-kernel#14824 · 1 comment ·
Maintainers usually reply within 3 days
-
help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
stack314/django-neolith#142 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
LedgerHQ/app-bitcoin-legacy#306 ·