scikit-learn/scikit-learn

FEAT Allow the vector-form representation of symetric distance matrices as input

Open

#29,133 opened on 2024年5月29日

GitHub で見る
 (8 comments) (0 reactions) (0 assignees)Python (27,020 forks)batch import
HardNew FeaturePerformancehelp wanted

Repository metrics

Stars
 (66,084 stars)
PR merge metrics
 (平均マージ 10d) (30d で 90 merged PRs)

説明

Describe the workflow you want to enable

I would like to calculate the upper triangle of a distance from scipy.spatial.distance.pdist instead of the redundant and memory intensive version from sklearn.metrics.pairwise_distances which has $0.5 * (N^2 - N)$ values and use this as input to metric="precompute"

Describe your proposed solution

from scipy.spatial.distance import pdist
X = # Data
dist = pdist(X, metric="jaccard") # Or any other but I just jaccard a lot for my boolean datasets.  Just didn't want to use euclidean/cosine 
clusterer = HDBSCAN(metric="precomputed_triangle")
clusterer.fit(dist)

In addition to the original functionality (or ideally replacing):

from scipy.spatial.distance import pdist, squareform
X = # Data
dist = squareform(pdist(X, metric="jaccard")) # Or any other but I just jaccard a lot for my boolean datasets.  Just didn't want to use euclidean/cosine 
clusterer = HDBSCAN(metric="precomputed")
clusterer.fit(dist)

Describe alternatives you've considered, if relevant

Using the redundant square form but this requires a lot more memory that isn't necessary.

Additional context

It may be worthwhile creating a DistanceMatrix object like https://scikit.bio/docs/latest/generated/skbio.stats.distance.DistanceMatrix.html#skbio.stats.distance.DistanceMatrix

コントリビューターガイド