[C API] Feature request: expose block-growth headroom for a dynamic index

Ouverte
  • Quelque chose ne va pas ?
#396 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 5 jours

Personne n'a encore pris cette issue.

Évaluation

Difficulté
3/5
Temps estimé
1-2 jours
Accessibilité débutants
72/100
Type d'issue
Fonctionnalité
Clarté
Clairement spécifiée
Activité
Active
Stack technique
c, cpp
Domaine
api

Piste de recherche

Start in bindings/c/include/svs/c/svs_c.h beside svs_index_get_memory_usage and svs_index_get_memory_breakdown, then trace the dynamic index implementation in dynamic_index.h and its resize/unsafe_resize paths. Add the read-only API and verify it reports capacity minus size for dynamic indices, returns zero for unsupported indices, and preserves the documented error-handling shape.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Problem

A dynamic index's memory reservation is tracked as committed capacity, not live occupancy. add_points grows data_/graph_/status_ in blocks (dynamic_index.h, resize/unsafe_resize calls), so a block often has spare rows left over after a resize. Committed capacity therefore overstates how many bytes an insert will actually consume.

An insert-admission gate that enforces a memory budget needs to know how much of that committed capacity is still unoccupied headroom, so it can allow an insert that would land in already-allocated spare rows and cost zero real bytes. Without that visibility, the gate must charge a fixed conservative growth cost per insert, and near the budget limit it refuses inserts that would not have grown memory at all.

What is needed is a cheap way to read how many rows of spare capacity a dynamic index currently has, without walking the graph.

Requested API

Add a new C API call, alongside the existing memory-usage family (svs_index_get_memory_usage, svs_index_get_memory_breakdown in bindings/c/include/svs/c/svs_c.h):

/// @brief Get the unused row capacity of a dynamic index before its next
/// internal block resize
/// @param index The index handle
/// @param out_vectors Pointer to store the number of rows of headroom
/// @param out_err An optional error handle to capture errors
/// @return true on success, false on failure
/// @remarks O(1). Reads capacity() - size() off the dynamic index's
/// blocked dataset. Returns 0 for indices that do not support dynamic
/// growth (e.g. static/read-only indices).
SVS_API bool svs_index_get_capacity_headroom(
    svs_index_h index, size_t* out_vectors, svs_error_h out_err /*=NULL*/
);
Why this shape
  • Mirrors the existing svs_index_get_memory_usage/svs_index_get_memory_breakdown signatures and doc-comment style, so it fits the current API family added for PR #364.
  • Returns a row count, not bytes. The caller already knows per-row byte cost from its own dimension/element-size config; a row count avoids duplicating SVS's internal block-size math on the caller side. A byte count works too, if that fits the existing memory-usage API family better; the caller can convert either way.
  • O(1): a capacity() - size() read, no graph walk.
Non-goals
  • Does not need to account for Deleted slots. delete_entry marks a slot Deleted, not Empty (dynamic_index.h), and add_points's reuse scan only matches Empty (dynamic_index.h:752). Only compact() turns Deleted back into usable capacity, so headroom from block growth and headroom from delete/compact are different numbers; this request is only for the former.
  • No behavior change to add_points, resize, or compact is being requested; this is a read-only accessor.
Langage dominant
C++
Étoiles
241
Forks
48
Merge moyen
4 j 21 h
PR mergées (30 j)
11

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de intel/ScalableVectorSearch

Toutes les issues de intel/ScalableVectorSearch

Issues similaires

Plus d'issues C++

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.