Complete index types, build controls and index introspection
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 25/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Bastante claro
- Estado de actividad
- Activo
- Área
- backend-api-design, database
Línea de trabajo
Start by mapping each operation to its public Rust method, type, and validation path at the pinned core revision 40f15ed1b71b66950961b175829fdf17a763f108; use the selected Rust core as the semantic reference. Review the existing C/C++ contracts and normal entry point tests, plus the linked work in #101, #55, and #153. Done means the remaining index and introspection capabilities have explicit compatibility semantics and coverage for build/query behavior, error and empty cases, and persisted/reopened data, with exact commands and validation limits recorded.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Current enums expose common vector and scalar indices, while pylance supports additional families and richer lifecycle introspection.
Required capabilities
- Add native NGram, ZoneMap and BloomFilter scalar-index paths, delivered in #138.
- Add supported RTree and MinHash LSH scalar-index paths and complete the family/FTS naming audit. MinHash LSH is delivered in #144. RTree #149 is deferred by the maintainer until an upstream release contains the confirmed reload fix, tracked in #153. The scalar-name audit records FM as native/PyO3-only rather than a target high-level scalar family.
- Add IVF_RQ and missing vector build/tuning controls through additive parameters or APIs. IVF_RQ=107 and complete native vector build options are delivered in #151. Both staged constructors share the trained DOT/precomputed-PQ scope protection, while definition-only subsets retain native behavior. Standalone trainer sample-rate/max-iterations controls and search effort are delivered in #158.
- Expose MinHash search with its documented distance semantics, delivered in #144.
- Add describe/statistics APIs, IVF model/centroid access and prewarm position/segment controls, delivered in #158.
- Cover index metadata merging and physical segment merging with explicit coverage and compatibility semantics. Native artifact merging and physical merge/commit/reopen are delivered in #158. Same-UUID metadata-file merge publication is delivered in #166 through caller-supplied coverage and the existing commit path. Its exact combined tree passed Kimi review, 710 ordinary Rust tests, all seven native consumers, static/shared CMake consumers and all seven hosted checks. C11/C++17 publish all three fragments, reopen the URI and verify 48 fixed phrase-query row IDs.
- Keep create/query/segment/prewarm/listing behavior and C++ wrappers consistent for each family.
Reuse #101 for its accepted segment-listing slice and #55 for distributed build coordination. Verify build/query results for each family, NULL/empty/error paths and persisted/reopened data. Version-specific training or optimizer changes depend on the core-baseline task.
Parent goal is #99. The source reference is pylance v14.0.2 at 40f15ed1b71b66950961b175829fdf17a763f108. Keep C/C++ contracts and meaningful normal-entry-point tests together. Record the exact source revision, commands, results, and validation limits.
Rust authority
The functional inventory uses pylance as a reference. The selected Lance Rust core at 40f15ed1b71b66950961b175829fdf17a763f108 defines available capabilities, parameter meanings, operation semantics and boundaries. C/C++ additions provide the foreign-language representation, ownership and error conversion needed to call that core.
Before implementation, map each operation to its public Rust method/type and validation path. Use direct Rust behavior as the semantic oracle. Any missing core operation is a core prerequisite. Binding-only product capabilities, policies and new query/retention semantics are outside this task. Existing public compatibility constraints must be handled explicitly during migration.
Rust operation limits
Dataset::merge_index_metadata supports Inverted in the target core. Other index families use merge_existing_index_segments and commit_existing_index_segments as applicable. Keep their distinct coverage, compatibility and error contracts. RTree depends on the core geo feature. Index type definitions and scalar parameters also live in lance-index-core at this baseline.
RTree release prerequisite
PR #149 is on hold by maintainer decision. #153 tracks the selected core's confirmed RTree reload false-negative bug, the merged upstream fix and the official-release adoption checks. Retain the current pin until that dependency is accepted, then rerun the preserved C/native indexed-versus-flat regression before resuming the PR. Other index-family and introspection work continues.
Family-route acceptance audit (2026-11-11)
The source/evidence audit at 27b2754652e886dc955502c18c02bf5517dbb8e0 found no functional defect in the non-held families. Existing shared create/query/segment/prewarm/listing routes, independent Rust semantic tests and real C11/C++17 consumers support the delivered checkboxes. This does not require a Cartesian product of every family, language and entry point. Retained #158/#166/#192/#198 execution evidence was matched to the relevant source; no new execution is claimed for the audit.
Native distinctions remain part of the contract: NGram/ZoneMap/BloomFilter prewarm is a no-op, so successful calls do not prove warmed cache entries. Their explicit scalar-segment paths fall back over the selected fragment domain. MinHash scalar-segment querying is rejected by native semantics. Metadata-file merging is Inverted-only; physical segment merge/commit remains a distinct operation. These are not missing binding accelerators.
The focused IVF_RQ/DOT coverage follow-up is accepted in #205, merged as aef913aa8d43e14bfa38889b1b5a4d2f7c206fe8. The C-ABI regression creates the index, queries it, closes/reopens the dataset, asserts independently calculated IDs and DOT distances, and compares with a native scanner on the persisted index. Both C queries assert that index partitions were loaded. Both partitions and all 64 candidate rows are refined, making this a deterministic DOT-route/persistence check rather than an approximate-recall benchmark. Existing L2/default coverage remains intact.
The final family-consistency checkbox remains open for the held RTree scope. Non-held family acceptance now includes the focused DOT follow-up. RTree #149/#153 is still excluded from delivered coverage, and FM retains the accepted native/PyO3-only inventory decision. The original issue scope is unchanged.
After the audit snapshot, ordinary index progress merged in #200 (8ba86c39c3fe83ff771df0aeab2fb21066e8930d) with separate native/consumer acceptance. That delivery closes #199; #205 subsequently resolves the DOT coverage follow-up. Neither lifts the RTree hold.
DOT follow-up execution and review
At 465742cf73b7b8ec34c415f976f26398e2844f80 on NieR with Rust 1.94.0, fmt, all-target check and Clippy passed. The exact DOT test passed (1 selected); the vector-index group passed (16 selected, including DOT); the legacy RQ default-bits test passed (1 selected). Independent source review found no blocker. The incoming #204 base delta does not change the index/scanner paths, test helper, dependencies or build configuration. Hosted Linux, MSRV, Clippy, format and Rustdoc passed; macOS and the additional review check were still pending when merged and are not claimed as successful execution. No new full native-consumer sweep is claimed for this test-only change.
- Lenguaje dominante
- Rust
- Estrellas
- 8
- Forks
- 17
- Merge medio
- 1 h 14 min
- PR fusionados (30 d)
- 89
Preparar el entorno
Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de lance-format/lance-c
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 15/100
lance-format/lance-c#153 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 35/100
lance-format/lance-c#103 · 4 comentarios ·
Los mantenedores suelen responder en 1 día
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 20/100
lance-format/lance-c#102 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 15/100
lance-format/lance-c#99 · 8 comentarios ·
Los mantenedores suelen responder en 1 día
Todos los issues de lance-format/lance-c
Issues similares
-
area:cli area:security bug priority:medium
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
rtk-ai/rtk#5127 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
status: new topic: ga4gh-sdk type: bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
Los mantenedores suelen responder en 1 día
-
comp-rbac
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
ClickHouse/ClickHouse#131875 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
Los mantenedores suelen responder en 1 día
-
`gws auth login` keeps the old token cache, so API calls use a token from the previous grantAbierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 80/100
googleworkspace/cli#984 ·
Los mantenedores suelen responder en 1 día