[C API] `svs_index_search_topk`: document the filtered-search contract for `filter_rate() <= 0`

未关闭
  • 哪里不对？
#415 3 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 5 天内回复

还没有人认领这个 Issue。

评估

难度
3/5
预计耗时
1-2 天
新手友好度
65/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
cpp
领域
backend-api-design

调研方向

Start with bindings/c/src/filtered_search.hpp and the filter_rate contract in svs_c.h; trace both early-exit sites described in the issue. Then inspect bindings/c/tests/ for the relevant search tests. Done means the tests cover both early exits and verify partial results, while the header documents the contract and the TODO is resolved.

由索引模型根据 Issue 内容生成。

描述

Summary

Please document what svs_index_search_topk does with an ID filter whose filter_rate() is <= 0 (or NULL). valkey-search relies on this behaviour, and the header doesn't state it.

The contract text in svs_c.h (lines 1297-1303 at 8285340) only covers filter_rate > 0.0: "If the actual observed filter hit rate is less than the provided filter_rate value, the function returns an empty result set." It says nothing about 0.0.

Current behaviour for filter_rate() <= 0

Code references are to main at 8285340.

  • estimate_initial_batch_size (bindings/c/src/filtered_search.hpp:92-97) skips the ID sampling and returns the batch hint, so the pre-search early exit (:185-190) cannot trigger.
  • hit_rate_sufficient (:65-69) returns true for filter_rate <= 0, so the in-loop exit that resets found = 0 (:218-222) cannot trigger either.
  • The search therefore runs until k filter-passing hits are found or the batch iterator is exhausted, and returns the hits found. That can be fewer than k, and is reported per query through offsets / total_results.

Ask

  1. Document the above in svs_c.h next to the existing filter_rate > 0.0 text, so it becomes part of the contract:
    • no sampling, and no early exit on the observed hit rate;
    • the hits found are returned, possibly fewer than k.
  2. Resolve the TODO at filtered_search.hpp:216-217 for the <= 0 case. A short comment stating the guarantee is enough.
  3. Add a test in bindings/c/tests/ asserting that, with filter_rate() returning 0.0 and a selective filter, a query returns a non-zero number of hits, all of which pass the filter.

No behaviour change is requested. Whether the positive-hint path should return partial results, or drop the ID sampling, is SVS's call. If it does change, an opt-in parameter that keeps today's default would avoid surprising other integrations.

Consumer context

valkey-search (ALGORITHM SVS_VAMANA) passes an ID filter built from query predicates and returns 0.0 from filter_rate. Its query planner already knows how many keys qualify before it calls SVS:

  • very selective filters (below roughly 0.001 * N qualifying keys) go to an exact brute-force path and never reach SVS;
  • for the rest, valkey-search sizes the first batch itself through the per-query search window (batch_hint = max(k, search_window_size) in bindings/c/src/index_vamana.cpp).

So valkey-search needs SVS to do no sampling or selectivity estimation, only to keep the <= 0 behaviour stable.

Scope change

This issue originally asked for partial results when the observed hit rate falls below a positive filter_rate. After the discussion below, valkey-search will handle selectivity on its own side, so the ask is narrowed to documenting the <= 0 contract. The ID sampler referenced in the original text has moved to bindings/c/src/index_vamana.cpp (around lines 78 and 140).

主要语言
C++
星标
241
派生
48
平均合并
4 天 21 小时
30 天内合并 PR
11

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

intel/ScalableVectorSearch 的其他 Issue

查看 intel/ScalableVectorSearch 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。