flaky: ruvector-quantization 'should perform search with binary quantization' recall floor fails on unseeded random vectors
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 84/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- typescript
- Domain
- testing
Research direction
Start with v3/@claude-flow/plugins/tests/ruvector-quantization.test.ts, especially the binary-quantization test at line 493 and the related perturbation case near line 508. Run npx vitest run __tests__/ruvector-quantization.test.ts, then make the recall check deterministic or base it on the planted near-duplicate scenario described in the issue. Done means the test passes reliably without masking regressions or depending on unseeded random vectors.
Written by the indexing model from the issue text.
Description
Observed: ruflo CI run 33974289751 (PR #3184 at 8b3927d, ubuntu Test Suite) — the CI test ratchet flagged one unexpected failing file, v3/@claude-flow/plugins/__tests__/ruvector-quantization.test.ts:
RuVector Quantization > Binary Quantization > should perform search with binary quantization
AssertionError: expected 0 to be greater than or equal to 0.1
at __tests__/ruvector-quantization.test.ts:493:22
The PR does not touch that file (diff vs main: LEDGER, gist, cli-cold-start test/bench, ADR status summary). The file is byte-identical to main, and locally it passes 28/28 (npx vitest run __tests__/ruvector-quantization.test.ts). It also passed on #3204's run 33972961703 minutes earlier on the same base.
Why it flakes: the test builds its corpus with randomVector(dimensions) (unseeded Math.random) and asserts binary-quantized top-k recall ≥ 0.1 against exact search. Binary quantization over i.i.d. random vectors has no such floor for small k — recall of 0 is a legitimate outcome of the RNG, not a regression. Sibling tests in the file have the same shape (line 508 perturbs with Math.random() too).
Ask (either):
- seed the generator (a small LCG/xorshift
randomVector(dim, seed)) so the corpus is fixed and the assertion is deterministic, or - assert a property binary quantization actually guarantees (e.g. recall over a corpus with a planted near-duplicate, as line 508 does), and drop the 0.1 floor on random data.
Until then the ratchet will keep failing unrelated PRs at random; I re-ran the failed job on #3184 rather than touching the baseline, since adding it to scripts/ci-test-baseline.txt would hide a real regression in that file.
🤖 Generated with RuFlo
- Dominant language
- TypeScript
- Stars
- 72.9k
- Forks
- 8.7k
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 74
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ruvnet/ruflo
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Similar issues
-
calcite-components needs triage refactor
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Esri/calcite-design-system#15203 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 91/100
-
community first-timers-only good first issue hacktoberfest help wanted low hanging fruit up-for-grabs
Difficulty 1/5 Under an hour Newbie friendliness 95/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Automattic/studio#4908 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100