[C API] `svs_index_search_topk`: document the filtered-search contract for `filter_rate() <= 0`

オープン
  • 何か問題がありますか？
#415 コメント 3 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 5 日以内に返信

まだ誰も着手していません。

評価

難易度
3/5
見積もり時間
1〜2日
初心者へのやさしさ
65/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
cpp
領域
backend-api-design

調査の方向性

Start with bindings/c/src/filtered_search.hpp and the filter_rate contract in svs_c.h; trace both early-exit sites described in the issue. Then inspect bindings/c/tests/ for the relevant search tests. Done means the tests cover both early exits and verify partial results, while the header documents the contract and the TODO is resolved.

索引モデルが issue の本文から書いたものです。

説明

Summary

Please document what svs_index_search_topk does with an ID filter whose filter_rate() is <= 0 (or NULL). valkey-search relies on this behaviour, and the header doesn't state it.

The contract text in svs_c.h (lines 1297-1303 at 8285340) only covers filter_rate > 0.0: "If the actual observed filter hit rate is less than the provided filter_rate value, the function returns an empty result set." It says nothing about 0.0.

Current behaviour for filter_rate() <= 0

Code references are to main at 8285340.

  • estimate_initial_batch_size (bindings/c/src/filtered_search.hpp:92-97) skips the ID sampling and returns the batch hint, so the pre-search early exit (:185-190) cannot trigger.
  • hit_rate_sufficient (:65-69) returns true for filter_rate <= 0, so the in-loop exit that resets found = 0 (:218-222) cannot trigger either.
  • The search therefore runs until k filter-passing hits are found or the batch iterator is exhausted, and returns the hits found. That can be fewer than k, and is reported per query through offsets / total_results.

Ask

  1. Document the above in svs_c.h next to the existing filter_rate > 0.0 text, so it becomes part of the contract:
    • no sampling, and no early exit on the observed hit rate;
    • the hits found are returned, possibly fewer than k.
  2. Resolve the TODO at filtered_search.hpp:216-217 for the <= 0 case. A short comment stating the guarantee is enough.
  3. Add a test in bindings/c/tests/ asserting that, with filter_rate() returning 0.0 and a selective filter, a query returns a non-zero number of hits, all of which pass the filter.

No behaviour change is requested. Whether the positive-hint path should return partial results, or drop the ID sampling, is SVS's call. If it does change, an opt-in parameter that keeps today's default would avoid surprising other integrations.

Consumer context

valkey-search (ALGORITHM SVS_VAMANA) passes an ID filter built from query predicates and returns 0.0 from filter_rate. Its query planner already knows how many keys qualify before it calls SVS:

  • very selective filters (below roughly 0.001 * N qualifying keys) go to an exact brute-force path and never reach SVS;
  • for the rest, valkey-search sizes the first batch itself through the per-query search window (batch_hint = max(k, search_window_size) in bindings/c/src/index_vamana.cpp).

So valkey-search needs SVS to do no sampling or selectivity estimation, only to keep the <= 0 behaviour stable.

Scope change

This issue originally asked for partial results when the observed hit rate falls below a positive filter_rate. After the discussion below, valkey-search will handle selectivity on its own side, so the ask is narrowed to documenting the <= 0 contract. The ID sampler referenced in the original text has moved to bindings/c/src/index_vamana.cpp (around lines 78 and 140).

主要言語
C++
スター
241
フォーク
48
平均マージ
4日 21時間
マージ済み PR（30日）
11

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

intel/ScalableVectorSearch のほかの issue

intel/ScalableVectorSearch の issue をすべて見る

似ている issue

C++ の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。