A-pythonPyTorchgood first issue
倉庫指標
- Star
- (6,582 star)
- PR 合併指標
- (平均合併 6天 1小時) (30 天內合併 219 個 PR)
描述
Currently the index training time is dominated by kmeans training, which can be sped up significantly via hardware acceleration.
Recently we added an API to take externally trained IVF centroids. So we could have an optional feature where the centroids are trained using pytorch or rapids then set the centroids explicitly.
Examples:
- https://pypi.org/project/kmeans-pytorch/
- https://medium.com/dropout-analytics/intro-to-k-means-clustering-with-cuml-b6d617e36456
https://github.com/lancedb/lance/blob/main/python/python/lance/dataset.py#L481