A-pythonPyTorchgood first issue
Repository metrics
- Stars
- (6,582 個のスター)
- PR merge metrics
- (平均マージ 6d 1h) (30d で 219 merged PRs)
説明
Currently the index training time is dominated by kmeans training, which can be sped up significantly via hardware acceleration.
Recently we added an API to take externally trained IVF centroids. So we could have an optional feature where the centroids are trained using pytorch or rapids then set the centroids explicitly.
Examples:
- https://pypi.org/project/kmeans-pytorch/
- https://medium.com/dropout-analytics/intro-to-k-means-clustering-with-cuml-b6d617e36456
https://github.com/lancedb/lance/blob/main/python/python/lance/dataset.py#L481