[C API] Feature request: expose block-growth headroom for a dynamic index

Aperta
  • Qualcosa non va?
#396 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 5 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
72/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
c, cpp
Ambito
api

Direzione di ricerca

Start in bindings/c/include/svs/c/svs_c.h beside svs_index_get_memory_usage and svs_index_get_memory_breakdown, then trace the dynamic index implementation in dynamic_index.h and its resize/unsafe_resize paths. Add the read-only API and verify it reports capacity minus size for dynamic indices, returns zero for unsupported indices, and preserves the documented error-handling shape.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Problem

A dynamic index's memory reservation is tracked as committed capacity, not live occupancy. add_points grows data_/graph_/status_ in blocks (dynamic_index.h, resize/unsafe_resize calls), so a block often has spare rows left over after a resize. Committed capacity therefore overstates how many bytes an insert will actually consume.

An insert-admission gate that enforces a memory budget needs to know how much of that committed capacity is still unoccupied headroom, so it can allow an insert that would land in already-allocated spare rows and cost zero real bytes. Without that visibility, the gate must charge a fixed conservative growth cost per insert, and near the budget limit it refuses inserts that would not have grown memory at all.

What is needed is a cheap way to read how many rows of spare capacity a dynamic index currently has, without walking the graph.

Requested API

Add a new C API call, alongside the existing memory-usage family (svs_index_get_memory_usage, svs_index_get_memory_breakdown in bindings/c/include/svs/c/svs_c.h):

/// @brief Get the unused row capacity of a dynamic index before its next
/// internal block resize
/// @param index The index handle
/// @param out_vectors Pointer to store the number of rows of headroom
/// @param out_err An optional error handle to capture errors
/// @return true on success, false on failure
/// @remarks O(1). Reads capacity() - size() off the dynamic index's
/// blocked dataset. Returns 0 for indices that do not support dynamic
/// growth (e.g. static/read-only indices).
SVS_API bool svs_index_get_capacity_headroom(
    svs_index_h index, size_t* out_vectors, svs_error_h out_err /*=NULL*/
);
Why this shape
  • Mirrors the existing svs_index_get_memory_usage/svs_index_get_memory_breakdown signatures and doc-comment style, so it fits the current API family added for PR #364.
  • Returns a row count, not bytes. The caller already knows per-row byte cost from its own dimension/element-size config; a row count avoids duplicating SVS's internal block-size math on the caller side. A byte count works too, if that fits the existing memory-usage API family better; the caller can convert either way.
  • O(1): a capacity() - size() read, no graph walk.
Non-goals
  • Does not need to account for Deleted slots. delete_entry marks a slot Deleted, not Empty (dynamic_index.h), and add_points's reuse scan only matches Empty (dynamic_index.h:752). Only compact() turns Deleted back into usable capacity, so headroom from block growth and headroom from delete/compact are different numbers; this request is only for the former.
  • No behavior change to add_points, resize, or compact is being requested; this is a read-only accessor.
Lingua principale
C++
Stelle
241
Fork
48
Merge medio
4g 21h
PR unite (30g)
11

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di intel/ScalableVectorSearch

Tutte le issue di intel/ScalableVectorSearch

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.