Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

[Feature] Support hybrid search with rrf, weighted_score and mrr rankers

Offen
  • Stimmt etwas nicht?
#407 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Anfängerfreundlichkeit
35/100
Issue-Typ
Feature
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
cpp
Bereich
databases, search

Rechercherichtung

Start with the Java HybridSearchBuilder and HybridSearchRanker design described in the issue, then inspect the existing table-level full-text search work in #404 and vector-search prerequisite. Implement the rankers, route validation, builder, and snapshot behavior described here, with tests for ranking, ties, weights, empty routes, validation, and snapshot pinning; primary-key support also depends on #410.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

enhancement
Search before asking
  • I searched in the issues and found nothing similar.
Motivation

Sub-issue of #399 (step 4: hybrid search).

Java Table#newHybridSearchBuilder() combines vector and full-text routes into one ranked result. It was built up over several PRs: apache/paimon#8271, apache/paimon#8288, apache/paimon#8294, apache/paimon#8324, apache/paimon#8327, apache/paimon#8336, apache/paimon#8348 and apache/paimon#8351. Paimon C++ has neither a hybrid search API nor a table-level vector search builder.

Java design:

  • Builder. HybridSearchBuilder (HybridSearchBuilderImpl) provides:

    • withPartitionFilter, withFilter
    • addVectorRoute(column, vector, limit[, weight[, options]]); the weight defaults to 1.0
    • addFullTextRoute(field, query, limit, weight)
    • withLimit
    • withRanker(String), withRrfRanker(), withWeightedScoreRanker()
    • executeLocal()

    All routes are pinned to one snapshot, and the filters are forwarded to every route. Full-text routes accept row filters since apache/paimon#9855.

  • Routes. HybridSearchRoute is either VECTOR or FULL_TEXT.

    • Weights must be finite and greater than 0.
    • Full-text routes reject route options.
    • The route builder rejects mixing vector and full-text settings.

  • Rankers. HybridSearchRanker normalizes the ranker name: blank means rrf; otherwise the name is trimmed and lower-cased.

    • rrf: Σ weight / (60 + rank + 1). Ties are broken by score descending, then row id ascending.
    • weighted_score: min-max normalize each route's scores to [0, 1] (all 1.0 when the range is 0), then compute Σ weight × normalized score.
    • mrr: Σ weight / (rank + 1).
    • The final top-k uses a heap.

  • Primary-key tables. They rank physical positions with PrimaryKeySearchRanker.weightedRrf, weightedScore and weightedMrr. Mixing primary-key physical results with global row-id results is rejected.

Solution
  1. Port the rankers and route validation. They are self-contained and can land first with unit tests.
  2. Add the hybrid search builder on top of the table-level full-text search (#404) and a table-level vector search builder.
  3. Add tests:
    • each ranker
    • tie-breaking
    • weights
    • an empty route result
    • route validation
    • snapshot pinning across routes
Anything else?
  • Prerequisite: a table-level vector search builder (Java VectorSearchBuilder / scan / read). It does not exist in Paimon C++ yet, and could be tracked separately or delivered as part of this issue.
  • Depends on #404. Primary-key hybrid search depends on #410.
Are you willing to submit a PR?
  • I'm willing to submit a PR!
Vorherrschende Sprache
C++
Sterne
65
Forks
31
Ø Merge
1 T. 23 Std.
Gemergte PRs (30 T.)
64

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus apache/paimon-cpp

Alle Issues in apache/paimon-cpp

Ähnliche Issues

Weitere Issues zu C++

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.