jev-v3 shadow week: review the disagreement report and decide the flip
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- bun, typescript
- Domain
- backend, observability, security
Research direction
Review the shadow week logs by running bun eval/live-report.ts --hours 168. Check for regressions where the live system denied but v3 would allow. Examine the 'headless blocks v3 would allow' list and outage bucket. Decide whether to flip the policy to 'jev-v3' based on the report, then update configuration accordingly.
Written by the indexing model from the issue text.
Description
Phase 2 of docs/plans/2026-09-19-intent-aware-judgment.md is merged (#100 #104 #101 #102 #110, with fixes #105 #108 #112 #114 #115). The jev-v3 judgment now runs in shadow on every fresh classification and logs v3 beside the live verdict. This issue is the plan's "Flip" step.
merged 2026-09-22 ─▶ OMP reloads the plugin ─▶ shadow week ─▶ review report ─▶ flip or not
(7 days) bun eval/live-report.ts --hours 168
Before starting the clock
- OMP has reloaded the plugin from
main(a decision line carriesv3). -
shadowV3is on (the default).
At the end of the week
- Run
bun eval/live-report.ts --hours 168. It must not print INCOMPLETE. - REGRESSIONS (denied live, v3 would allow) is empty. That is the plan's flip criterion.
- Read the "headless blocks v3 would allow" list and the outage bucket.
- Decide the flip: set
policy: "jev-v3", then after a further week without regression, remove the jev-v2 path.
Known limits going in
- Branch 5 (the reviewer) is UNSURE until Phase 3 builds it.
- A task stated mid-session still ages out of evidence (#106).
- The live evidence collectors read past
/clear(#103); the jev-v3 builder doesn't. - The shadow costs two more Jev requests per fresh classification.
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 1
- Avg merge
- 2h 10m
- Merged PRs (30d)
- 64
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from STRML/omp-classifier
-
Decide whether a coordinator may lift a headless worker's refusal (the trust boundary #68 defers)Openenhancement ready-for-human
Difficulty 5/5 Over a week Newbie friendliness 25/100
STRML/omp-classifier#142 ·
Maintainers usually reply within 1 day
-
enhancement ready-for-human
Difficulty 5/5 Over a week Newbie friendliness 35/100
STRML/omp-classifier#13 · 6 comments ·
Maintainers usually reply within 1 day
All issues in STRML/omp-classifier
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
external-issue to-triage
Difficulty 1/5 Under an hour Newbie friendliness 90/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
diegosouzapw/OmniRoute#15401 ·
Maintainers usually reply within 2 days
-
Sign the pledgeOpen
Difficulty 1/5 Under an hour Newbie friendliness 95/100
input-output-hk/devx-updates#163 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
code-yeongyu/oh-my-openagent#9454 ·
Maintainers usually reply within 1 day