I have used correlation metric as distance measure for hierachical clustering and obtained the clusters. I used scikit (python 3.5) for clustering hierachical cluster.

Now, I want to use another clustering algorithm with same dataset.

I am not sure whether any other clustering algorithms will take correlation as distance metric.

I request you to assist with which algorithms are feasible for this ? and How can I use it.

  • $\begingroup$ Almost every algorithm except k-means and GMM. Have a look at ELKI, it includes correlation (maybe even with index for acceleration!) and several clustering algorithms such as DBSCAN, OPTICS, HDBSCAN*, that can be used with correlation distance. $\endgroup$ – Has QUIT--Anony-Mousse Apr 25 '17 at 19:37
  • $\begingroup$ You are right. Those are implemented in JAVA. Can you assist me with Python implementations $\endgroup$ – Abhishek Apr 26 '17 at 7:59

From scikit-learn, any clustering that can work with "precomputed" distances should work. For example, Affinity Propagation or DBSCAN.

(To be clear, you have to first calculate the pairwise distance matrix between your data points and then use this matrix as input)

| cite | improve this answer | |
  • $\begingroup$ Thanks Leonardo, So instead of dataset I should give correlation(distance metric) as input ? $\endgroup$ – Abhishek Apr 25 '17 at 11:24
  • $\begingroup$ Yes, first you use dist=sklearn.metrics.pairwise.pairwise_distances(data) to calculate the distance matrix from your data, and then you use the resulting dist object as input to the clustering algorithms, remembering to select the option affinity="precomputed for affinity propagation or metric="precomputed" in the case of DBSCAN. BTW for affinity propagation I think you need to transform the distances into similarities. $\endgroup$ – Leo Martins Apr 25 '17 at 12:00
  • $\begingroup$ dist=sklearn.metrics.pairwise.pairwise_distances(data) Does this correlate dataset with each other ? $\endgroup$ – Abhishek Apr 26 '17 at 8:03
  • $\begingroup$ Each element of your data set with each other, yes. Suppose your data set are N rows (samples) with M columns (dimensions, features). then sklearn.metrics.pairwise.pairwise_distances(data, metric="correlation") will return a matrix of size NxN with the correlation distances between the elements. (I assumed this is how you obtained the distances inthe first place.) $\endgroup$ – Leo Martins Apr 26 '17 at 9:32
  • $\begingroup$ Thanks Leo,My dataset is of 1000 variables. and each dataset is measured for 50,000 time intervals. So dataset looks like matrix of 50,000 rows and 1000 columns $\endgroup$ – Abhishek Apr 26 '17 at 9:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.