QCFilterTransform should have richer syntax for describing tests and outputs

Open
#398 5 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
12/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
python, yaml
Domain
data

Research direction

The current behaviour lives in QCFilterTransform, whose argument is a comma-separated string of var:lower:upper specs; start by reading how that string is parsed. The issue proposes a YAML dict of named tests (higher_than, lower_than, does_not_contain and others) with per-test messages and max_times, but the test names, the dict schema and the new class name are all still open questions. The thread has been idle for about two years, so agree on a design with a maintainer before writing any code.

Written by the indexing model from the issue text.

Description

enhancement

And maybe shouldn't be called QCFilterTransform

Args are currently limited to a string of comma-separated var:lower:upper specs. Would be much more powerful if it took a dict such that you could specify:

kwargs:
  fields:
    RTMP:
      higher_than:
        value: -40
        message: 'Temps too low: {RTMP}'
      lower_than:
        value: 40
        message: 'Temps too high: {RTMP}'
      does_not_contain:
        value 'NaN'
        message: 'Received NaN for RTMP'
        max_times: 5

Could be called TestValueTransform (?) and could have tests like

  • lower_than
  • higher_than
  • lower_than_or_equal
  • higher_than_or_equal
  • contains (with a 'negate' arg?, or the following)
  • does_not_contain
  • matches_regex

and...?

Dominant language
Python
Stars
57
Forks
30
Avg merge
3h 14m
Merged PRs (30d)
56

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from OceanDataTools/openrvdas

All issues in OceanDataTools/openrvdas

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.