Métricas do repositório
- Stars
- (10.303 estrelas)
- Métricas de merge de PR
- (Mesclagem média 2d 17h) (82 fundiu PRs em 30d)
Description
Hello, your friendly neighborhood computational chemist here!
I'd like to be able to perform vector searches over binary / boolean fields (in this case, molecular fingerprints e.g., https://chemicbook.com/2021/03/25/a-beginners-guide-for-understanding-extended-connectivity-fingerprints.html).
but it won't let me ->
ValueError: LanceError(IO): Column fingerprint is not a vector column (type: FixedSizeList(Field { name: "item", data_type: Boolean, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, 2048)), /home/runner/work/lance/lance/rust/lance/src/dataset/scanner.rs:384:35
import pyarrow as pa
import lancedb
from lancedb.pydantic import LanceModel, Vector
class Molecule(LanceModel):
id: str
smiles: str
fingerprint: Vector(2048, pa.bool_())
uri = "./out/strict_fragments.lancedb"
db = lancedb.connect(uri)
tbl = db.create_table(
"molecules",
schema = Molecule,
mode = 'overwrite'
)
Neither does it work with any flavor of integer. It would be amazing to get this working because the sparse booleans obviously take up much less room on disk. These fingerprints are often very sparse, so it would be even better if we could use a sparse format like SparseTensor in Arrow. In addition, Jaccard is the preferred distance metric, but Cosine or inner product are preferable to euclidean.