scikit-learn/scikit-learn
GitHub で見るFEAT Allow the vector-form representation of symetric distance matrices as input
Open
#29,133 opened on 2024年5月29日
HardNew FeaturePerformancehelp wanted
Repository metrics
- Stars
- (66,084 stars)
- PR merge metrics
- (平均マージ 10d) (30d で 90 merged PRs)
説明
Describe the workflow you want to enable
I would like to calculate the upper triangle of a distance from scipy.spatial.distance.pdist instead of the redundant and memory intensive version from sklearn.metrics.pairwise_distances which has $0.5 * (N^2 - N)$ values and use this as input to metric="precompute"
Describe your proposed solution
from scipy.spatial.distance import pdist
X = # Data
dist = pdist(X, metric="jaccard") # Or any other but I just jaccard a lot for my boolean datasets. Just didn't want to use euclidean/cosine
clusterer = HDBSCAN(metric="precomputed_triangle")
clusterer.fit(dist)
In addition to the original functionality (or ideally replacing):
from scipy.spatial.distance import pdist, squareform
X = # Data
dist = squareform(pdist(X, metric="jaccard")) # Or any other but I just jaccard a lot for my boolean datasets. Just didn't want to use euclidean/cosine
clusterer = HDBSCAN(metric="precomputed")
clusterer.fit(dist)
Describe alternatives you've considered, if relevant
Using the redundant square form but this requires a lot more memory that isn't necessary.
Additional context
It may be worthwhile creating a DistanceMatrix object like https://scikit.bio/docs/latest/generated/skbio.stats.distance.DistanceMatrix.html#skbio.stats.distance.DistanceMatrix