[C API] `svs_index_search_topk`: document the filtered-search contract for `filter_rate() <= 0`

Aberta
  • Algo errado?
#415 3 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 5 dias

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
3/5
Tempo estimado
1-2 dias
Facilidade para iniciantes
65/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
cpp
Domínio
backend-api-design

Direção de pesquisa

Start with bindings/c/src/filtered_search.hpp and the filter_rate contract in svs_c.h; trace both early-exit sites described in the issue. Then inspect bindings/c/tests/ for the relevant search tests. Done means the tests cover both early exits and verify partial results, while the header documents the contract and the TODO is resolved.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Summary

Please document what svs_index_search_topk does with an ID filter whose filter_rate() is <= 0 (or NULL). valkey-search relies on this behaviour, and the header doesn't state it.

The contract text in svs_c.h (lines 1297-1303 at 8285340) only covers filter_rate > 0.0: "If the actual observed filter hit rate is less than the provided filter_rate value, the function returns an empty result set." It says nothing about 0.0.

Current behaviour for filter_rate() <= 0

Code references are to main at 8285340.

  • estimate_initial_batch_size (bindings/c/src/filtered_search.hpp:92-97) skips the ID sampling and returns the batch hint, so the pre-search early exit (:185-190) cannot trigger.
  • hit_rate_sufficient (:65-69) returns true for filter_rate <= 0, so the in-loop exit that resets found = 0 (:218-222) cannot trigger either.
  • The search therefore runs until k filter-passing hits are found or the batch iterator is exhausted, and returns the hits found. That can be fewer than k, and is reported per query through offsets / total_results.

Ask

  1. Document the above in svs_c.h next to the existing filter_rate > 0.0 text, so it becomes part of the contract:
    • no sampling, and no early exit on the observed hit rate;
    • the hits found are returned, possibly fewer than k.
  2. Resolve the TODO at filtered_search.hpp:216-217 for the <= 0 case. A short comment stating the guarantee is enough.
  3. Add a test in bindings/c/tests/ asserting that, with filter_rate() returning 0.0 and a selective filter, a query returns a non-zero number of hits, all of which pass the filter.

No behaviour change is requested. Whether the positive-hint path should return partial results, or drop the ID sampling, is SVS's call. If it does change, an opt-in parameter that keeps today's default would avoid surprising other integrations.

Consumer context

valkey-search (ALGORITHM SVS_VAMANA) passes an ID filter built from query predicates and returns 0.0 from filter_rate. Its query planner already knows how many keys qualify before it calls SVS:

  • very selective filters (below roughly 0.001 * N qualifying keys) go to an exact brute-force path and never reach SVS;
  • for the rest, valkey-search sizes the first batch itself through the per-query search window (batch_hint = max(k, search_window_size) in bindings/c/src/index_vamana.cpp).

So valkey-search needs SVS to do no sampling or selectivity estimation, only to keep the <= 0 behaviour stable.

Scope change

This issue originally asked for partial results when the observed hit rate falls below a positive filter_rate. After the discussion below, valkey-search will handle selectivity on its own side, so the ask is narrowed to documenting the <= 0 contract. The ID sampler referenced in the original text has moved to bindings/c/src/index_vamana.cpp (around lines 78 and 140).

Linguagem predominante
C++
Estrelas
241
Forks
48
Merge médio
4d 21h
PRs com merge (30d)
11

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de intel/ScalableVectorSearch

Todas as issues de intel/ScalableVectorSearch

Issues semelhantes

Mais issues de C++

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.