[Bug] Primary-key vector search full/detail modes can surface superseded rows for first-row tables without deletion vectors
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 48/100
Direzione di ricerca
Reproduce the first-row primary-key table scenario with deletion vectors disabled and inspect PrimaryKeyVectorBucketSearch, PrimaryKeyIndexedSplitRead, and RawFileSplitRead. Trace how exact-scan positions are read without merge-on-read deduplication, then review the surrounding vector-search tests and related issue #9894. Done means full and detail searches no longer return superseded rows.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Search before asking
- I searched in the issues and found nothing similar.
Paimon version
master
Compute Engine
Any (primary-key table vector search)
Minimal reproduce step
On a first-row merge-engine table with deletion vectors disabled and a pk-vector index, write a key, let it compact, then write the same key again (a version first-row drops on read). Run a vector search with vector-index.search-mode set to full or detail.
What doesn't meet your expectations?
The vector search can return the superseded row version, which a normal merge-on-read query would never return.
Root cause: the exact scan in PrimaryKeyVectorBucketSearch returns physical row positions per file with no dedup by primary key, and the readback (PrimaryKeyIndexedSplitRead / RawFileSplitRead) reads those positions directly without a merge-on-read pass. first-row resolves duplicate keys only through the merge/compaction path, so when a superseded version has not yet been merged away it is physically present and the exact scan can surface it. This is specific to the first-row engine with deletion vectors disabled; deletion-vector-enabled tables mark superseded rows and exclude them from the exact scan.
This predates #9894. That change extends the exact scan to level-0 APPEND files, which makes the surfacing more frequent (a fresh superseded write is the common case), but the underlying gap already applied to superseded versions promoted into compacted files.
Anything else?
One direction would be to apply a primary-key dedup to the exact-scan candidates before ranking (or require deletion vectors for full/detail on primary-key vector search). Filed for tracking, not blocking #9894.
Are you willing to submit a PR?
- I'm willing to submit a PR!
- Lingua principale
- Java
- Stelle
- 3.4k
- Fork
- 1.4k
- Merge medio
- 1g 14h
- PR unite (30g)
- 468
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di apache/paimon
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
[Bug] [Hive] IndexOutOfBoundsException when converting an unavailable dynamic BETWEEN predicate Apertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 84/100
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
Tutte le issue di apache/paimon
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
infinispan/infinispan#18150 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 84/100
-
untriaged
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100