[C API] Feature request: expose block-growth headroom for a dynamic index

Aberta
  • Algo errado?
#396 1 comentário 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 5 dias

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
3/5
Tempo estimado
1-2 dias
Facilidade para iniciantes
72/100
Tipo de issue
Funcionalidade
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
c, cpp
Domínio
api

Direção de pesquisa

Start in bindings/c/include/svs/c/svs_c.h beside svs_index_get_memory_usage and svs_index_get_memory_breakdown, then trace the dynamic index implementation in dynamic_index.h and its resize/unsafe_resize paths. Add the read-only API and verify it reports capacity minus size for dynamic indices, returns zero for unsupported indices, and preserves the documented error-handling shape.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Problem

A dynamic index's memory reservation is tracked as committed capacity, not live occupancy. add_points grows data_/graph_/status_ in blocks (dynamic_index.h, resize/unsafe_resize calls), so a block often has spare rows left over after a resize. Committed capacity therefore overstates how many bytes an insert will actually consume.

An insert-admission gate that enforces a memory budget needs to know how much of that committed capacity is still unoccupied headroom, so it can allow an insert that would land in already-allocated spare rows and cost zero real bytes. Without that visibility, the gate must charge a fixed conservative growth cost per insert, and near the budget limit it refuses inserts that would not have grown memory at all.

What is needed is a cheap way to read how many rows of spare capacity a dynamic index currently has, without walking the graph.

Requested API

Add a new C API call, alongside the existing memory-usage family (svs_index_get_memory_usage, svs_index_get_memory_breakdown in bindings/c/include/svs/c/svs_c.h):

/// @brief Get the unused row capacity of a dynamic index before its next
/// internal block resize
/// @param index The index handle
/// @param out_vectors Pointer to store the number of rows of headroom
/// @param out_err An optional error handle to capture errors
/// @return true on success, false on failure
/// @remarks O(1). Reads capacity() - size() off the dynamic index's
/// blocked dataset. Returns 0 for indices that do not support dynamic
/// growth (e.g. static/read-only indices).
SVS_API bool svs_index_get_capacity_headroom(
    svs_index_h index, size_t* out_vectors, svs_error_h out_err /*=NULL*/
);
Why this shape
  • Mirrors the existing svs_index_get_memory_usage/svs_index_get_memory_breakdown signatures and doc-comment style, so it fits the current API family added for PR #364.
  • Returns a row count, not bytes. The caller already knows per-row byte cost from its own dimension/element-size config; a row count avoids duplicating SVS's internal block-size math on the caller side. A byte count works too, if that fits the existing memory-usage API family better; the caller can convert either way.
  • O(1): a capacity() - size() read, no graph walk.
Non-goals
  • Does not need to account for Deleted slots. delete_entry marks a slot Deleted, not Empty (dynamic_index.h), and add_points's reuse scan only matches Empty (dynamic_index.h:752). Only compact() turns Deleted back into usable capacity, so headroom from block growth and headroom from delete/compact are different numbers; this request is only for the former.
  • No behavior change to add_points, resize, or compact is being requested; this is a read-only accessor.
Linguagem predominante
C++
Estrelas
241
Forks
48
Merge médio
4d 21h
PRs com merge (30d)
11

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de intel/ScalableVectorSearch

Todas as issues de intel/ScalableVectorSearch

Issues semelhantes

Mais issues de C++

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.