[Bug] Precise union drops repeated keys in the SortShuffle path
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Idoneità per principianti
- 75/100
Direzione di ricerca
Read UnionFilterIterator.java, especially NoRepeatValueIterator.hasNext() and saveElement(), then trace how the reducer handles comparator-equal groups and EOF. Reproduce with 20,000 keys occurring twice and check behavior across flush boundaries and repeated probes. Done means each matching name group emits one representative while preserving the intended representative/value rule.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Bug Type
Logic / missing results in precise union queries.
Environment
Development source reviewed in PR #3395, Java 17. Reproduced with the production iterators and SortShuffle in a standalone JVM; no running Store cluster was used.
Expected and actual behavior
Once UnionFilterIterator has more than 10,000 map entries, it selects the SortShuffle/reducer path. NoRepeatValueIterator omits an entire comparator-equal group when that group occurs more than once, rather than returning one representative.
A probe with 20,000 keys, each read twice returns 0 rows; expected 20,000 rows, one per key. Each occurrence is a newly allocated BackendColumn with identical name/value bytes for that key.
The same query shape should retain each overlapping key once. MapUnionIterator already does so on the alternate small-query path.
Reproduction
- Generate keys as four-byte integer arrays and two occurrences per key, with fresh BackendColumn objects and arrays for every row.
- Populate an IntersectionWrapper from the first pass and call
proc(); all keys in this dataset are repeated candidates. - Construct UnionFilterIterator over an independent second pass, using the production BackendColumn serializer and name comparator.
- Consume normally with
while (iterator.hasNext()) { iterator.next(); count++; }. The observed count is zero.
The 10,000 threshold selects the union's sorting branch. SortShuffle has its own higher physical-file threshold, so this defect can occur before it writes an actual spill file; its file-backed path uses the same reducer.
Cause and focused follow-up
saveElement() preserves repeated candidates for sorting, but NoRepeatValueIterator.hasNext() skips groups where its duplicate counter is positive, including the final group. Fix reduction to emit a representative once for each matching name group, and cover repeated keys across flush boundaries, repeated probes, and EOF. Preserve the intended representative/value rule.
Sources: UnionFilterIterator and reducer, MapUnion alternate query path.
This is independent of the in-memory map-key defect tracked in https://github.com/apache/hugegraph/issues/3396 and predates the changed iterator-state logic in PR #3395. Keep it as a separate correctness fix rather than expanding that PR.
Issue search performed for NoRepeatValueIterator and UnionFilterIterator; no matching existing issue was found.
- Lingua principale
- Java
- Stelle
- 3.2k
- Fork
- 642
- Merge medio
- 1g 12h
- PR unite (30g)
- 105
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di apache/hugegraph
-
[Bug] PD task REST: balanceLeaders throws a bare HTTP 500 during the balance-shard window, and task responses cannot be told apart from a no-op or a follower answerForse già presa @pallavithegod l’ha presa 8 giorni fa. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
apache/hugegraph#3231 · 2 commenti · 1 assegnatario ·
I maintainer di solito rispondono entro 1 giorno
-
[Bug] Gremlin sandbox blocks HStore gRPC allocator configuration readsForse già presa @contrueCT l’ha presa 1 giorno fa. Aperta
Difficoltà 4/5 3-5 giorni Idoneità per principianti 25/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 55/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
I maintainer di solito rispondono entro 1 giorno
-
[Bug] PD TTL cleanup deletes the marker-shaped name instead of the expired key, so expired values stay in the store until readForse già presa @bitflicker64 l’ha presa 7 giorni fa. Aperta
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di apache/hugegraph
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
strimzi/strimzi-kafka-operator#13328 · 1 commento · 3 reazioni ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 67/100
OpenLiberty/ci.maven#2137 ·
I maintainer di solito rispondono entro 1 giorno
-
Payer adjustment can exceed the balance left after copay and deductible, so claim payments add up to more than the chargeForse già presa @DG47 l’ha presa oggi. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 79/100
synthetichealth/synthea#1772 ·
I maintainer di solito rispondono entro 13 giorni
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
geonetwork/core-geonetwork#9592 ·
I maintainer di solito rispondono entro 2 giorni
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
NationalSecurityAgency/ghidra#10040 ·
I maintainer di solito rispondono entro 2 giorni