"vector_top_k" with duplicated vectors

Open
#2,104 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 11 days

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
c, sqlite
Domain
databases, search

Research direction

Reproduce the behavior with duplicated embeddings, comparing vector_top_k without an index against libsql_vector_idx. Inspect those entry points to determine how duplicate vectors are handled; done means every matching row is returned when embeddings are identical, without requiring added noise.

Written by the indexing model from the issue text.

Description

Hey everyone!

I have a little database that I use for semantic search.

I only embed a subset of the text which means that some embedding can be duplicated.

Without the index, I can access them without any issue, but with libsql_vector_idx, only one of the row is returned. I guess that only one rowid can be stored per vector.

Any idea how I can solve that (except adding a little bit of noise?)

Dominant language
C
Stars
17.3k
Forks
562
Avg merge
10d 15h
Merged PRs (30d)
1

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from tursodatabase/libsql

All issues in tursodatabase/libsql

Similar issues

More C issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.