Flaky CI: intermittent tests-without-auth failures (boost timeDecay ~1-in-6, plus two confirmed serialization bugs)
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 35/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- docker, node.js, typescript
- Ambito
- backend-api-design, ci-cd, testing-qa
Direzione di ricerca
Inizia da test/collections/query/integration.test.ts e dai test falliti descritti nell’issue, quindi esamina src/collections/serialize/index.ts, src/collections/deserialize/index.ts e test/collections/journey.test.ts. Il lavoro è completo quando il boost test è deterministico, decay raggiunge il proto generato, l’output di debug è stato rimosso, UUIDs mancanti e un setup fallito non causano errori di Cleanup opachi e CI non presenta più flakiness per questi casi.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Summary
tests-without-auth fails intermittently across all Weaviate versions and both Node versions, on unrelated PRs. Re-running the identical commit turns it green, so the code under test is not the cause.
Most recent evidence: run 33513846023 failed, then passed on all 8 jobs after a re-run with no code change.
At least three distinct flakes are involved. One is now understood in detail; the other two are not.
1. query > boost > 'timeDecay' — understood, ~1-in-6 failure rate
test/collections/query/integration.test.ts
AssertionError: expected [ …(3) ] to not deeply equal [ …(3) ]
The assertion is:
const baseline = ids(await collection.query.nearVector([1, 2, 3]));
const boosted = ids(await collection.query.nearVector([1, 2, 3], { boost }));
expect(boosted).not.toEqual(baseline);
Measured behaviour
Reproduced locally against semitechnologies/weaviate:1.38.2, rebuilding the collection each trial:
- 5/30 trials (16.7%) failed on unmodified
main. - Across rebuilds the boosted ordering spans all six permutations of the three objects, near-uniformly. 1/6 = 16.7%, matching the observed rate.
- Within a single collection, both baseline and boosted are perfectly stable (0/30 unstable). This is why re-running the same query never reproduces it — only rebuilding the collection does.
Ruled out
| Hypothesis | Result |
|---|---|
All three createdAt set to the same Date.now() |
ruled out — spreading them failed at the same rate |
blue/green share vector [1,2,3] (distance tie) |
ruled out — distinct vectors failed at the same rate |
Query racing vector-index build after insertMany |
ruled out — 250ms and 1000ms waits changed nothing |
Assessment
The test's premise is the problem: it assumes a boost must reorder three near-tied results, which the server does not guarantee. Retuning the input values will not hold — 467e5a7b already attempted exactly that in July, and this is the same test failing again.
Suggested direction: assert something deterministic (an expected ordering over well-separated data, or scores/membership) rather than "the order changed". The other three boost cases in the same it.each (filter, numericDecay, numericProperty) have never flaked.
2. Two confirmed bugs found while investigating
These are independent of the flake and are separately actionable.
2a. decay is silently dropped from Boost.timeDecay / Boost.numericDecay
The public type uses decay; the generated proto field is decayValue:
src/collections/types/query.ts:211,223—decay?: numbersrc/proto/v1/search_get.ts:351-367—decayValue?: numbersrc/collections/serialize/index.ts:1645,1659—fromPartial({ ...timeDecay, curve })
fromPartial discards the unknown decay key, so the value never reaches the wire. Observed payload:
{"timeDecay":{"property":"createdAt","origin":"2024-01-01T00:00:00Z","scale":"35d","curve":3}}
With a two-line mapping (decayValue: decay) it becomes:
{"timeDecay":{"property":"createdAt","origin":"...","scale":"35d","curve":3,"decayValue":0.2}}
Any user passing decay today has it ignored, with no error. Note: applying this fix did not make the ordering deterministic, so it is not the cause of the flake above.
2b. Debug logging shipped in the client
src/collections/serialize/index.ts:1679
console.error(JSON.stringify(out));
Inside Serialize.boostGRPC — writes to stderr on every boosted query.
3. Not yet understood
3a. data > dryRun deleteMany — TypeError: Stringified UUID is invalid
❯ stringify node_modules/uuid/dist-node/stringify.js:31:15
❯ src/collections/deserialize/index.ts:536:19 id: stringify(obj.uuid),
The server returns a BatchDeleteObject with no uuid; the proto defaults it to new Uint8Array(0) (src/proto/v1/batch_delete.ts:303,332), unsafeStringify then indexes byteToHex[undefined] and produces an invalid string. Not reproduced locally; why the server omits the field is unresolved. Regardless, the call site is unguarded and turns this into an opaque TypeError.
3b. collections/journey.test.ts — fetch failed in beforeAll
WeaviateStartUpError: Weaviate failed to startup with message: fetch failed (src/connection/helpers.ts:123)
This suite connects to a fixed external WCD cluster rather than the CI container, so all seven matrix jobs hit the same shared cluster concurrently and a transient network failure kills the suite (observed with zero failed assertions: 1204 passed | 83 skipped).
Separately, afterAll at test/collections/journey.test.ts:16 is unguarded:
afterAll(() => client.collections.delete(collectionName));
When beforeAll fails, client is undefined and this throws a second, misleading TypeError: Cannot read properties of undefined (reading 'collections'), obscuring the real cause. A client?. guard would fix that much.
Worth considering whether a shared, externally-mutable cluster belongs in the blocking CI path at all — an upgrade to it recently reddened every open PR simultaneously (fixed in #469).
- Lingua principale
- TypeScript
- Stelle
- 101
- Fork
- 43
- Merge medio
- 7h 31m
- PR unite (30g)
- 6
Preparare l'ambiente
Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di weaviate/typescript-client
-
CI: journey test asserts WCD cluster's replication default; timeDecay boost test is flakyForse già presa @dudanogueira l’ha presa 49 giorni fa. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
weaviate/typescript-client#485 ·
I maintainer di solito rispondono entro 1 giorno
-
data.ingest() silently stores empty objects for the NonReferenceInputs form its type signature acceptsForse già presa @OsamaAnsar l’ha presa 79 giorni fa. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
weaviate/typescript-client#456 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
weaviate/typescript-client#411 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 3/5 1-2 giorni Idoneità per principianti 54/100
weaviate/typescript-client#487 ·
I maintainer di solito rispondono entro 1 giorno
-
Add support for new generative-meta moduleForse già presa @OsamaAnsar l’ha presa 64 giorni fa. Aperta
Difficoltà 3/5 1-2 giorni Idoneità per principianti 68/100
weaviate/typescript-client#478 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di weaviate/typescript-client
Issue simili
-
bot:ai-assisted component:platform-js status:untriaged
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
midnightntwrk/platform-js#69 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
Picolab/pico-engine#668 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
I maintainer di solito rispondono entro 1 giorno
-
ai-driven-qa bug claude Cosmetic
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
I maintainer di solito rispondono entro 1 giorno
-
needs verification
Difficoltà 2/5 1-3 ore Idoneità per principianti 64/100
I maintainer di solito rispondono entro 13 giorni