0
$\begingroup$

Can anyone please let me know once I have a distance matrix at hand can any clustering method be used on it regardless of the type of distance measure used to get the matrix. Can the distance matrix be thought of as a new dataset and do the clustering? Is it going to give different results?

$\endgroup$
  • 1
    $\begingroup$ I don't know if every clustering method will work but most should. Yes, you can think of it like a new dataset. Different metrics will give different results. $\endgroup$ – user2974951 Dec 7 '18 at 7:11
  • $\begingroup$ It depends on how you perform the clustering. What exactly are you proposing? $\endgroup$ – whuber Dec 7 '18 at 13:44
  • $\begingroup$ I'm clustering time series data. So I developed a distance matrix that suits for time series and fed ut to the clustering functions in r. I am aware that I can use tclust for time series but the problem is the excessive consumption of time. By using the distance matrix this can be done at a considerable rate. What do you suggest on this? $\endgroup$ – ap123 Dec 10 '18 at 5:47
  • $\begingroup$ What I thought to do was use the made distance matrix as the input raw data as it is relative to the original data. And when in cases like kmeans when calculating the mean the distance matrix will give a relatively similar measure since the matrix represents the variability in the actual da. Please correct me if I am wrong. $\endgroup$ – ap123 Dec 10 '18 at 5:47
2
$\begingroup$

Many, if not most, clustering methods can be implemented using a distance matrix as input. In skleaen and R, most functions will accept a distance matrix, or can be configured to do so.

The main reason to not use distance matrixes is scalability: a distance maid needs O(n²) memory and O(n²) time to build. So at around 30.000 to 60.000 instances you usually run into problems.

There are some obvious exceptions: k-means for example needs to compute the mean, which requires the original data. It also never uses the distance inbetween of two data points, only point to center.

If you falsely pass a distance matrix to k-means you will likely still get a possible result. Just the mean will no longer be in your input data, and you get some very hard to grasp semantic - the presence of absence of some big cluster in one region affects the clustering result in a complete different region. So statistically the result is not desirable, even if it at first sight appears to be okay.

$\endgroup$
  • $\begingroup$ Thank you for the detailed answer. I'm using R in my clustering and I see your point. The problem is I'm clustering time series data. So I developed a distance matrix that suits for time series and fed ut to the clustering functions in r. I am ware that I can use tclust for time series but the problem is the excessive consumption of time. By using the distance matrix this can be done at a considerable rate. What do you suggest on this? $\endgroup$ – ap123 Dec 10 '18 at 5:37
  • $\begingroup$ What I thought to do was use the made distance matrix as the input raw data as it is relative to the original data. And when in cases like kmeans when calculating the mean the distance matrix will give a relatively similar measure since the matrix represents the variability in the actual da. Please correct me if I am wrong. $\endgroup$ – ap123 Dec 10 '18 at 5:45
  • $\begingroup$ Well, k-means is one of the few methods that do not allow a distance matrix, for a reason. Why don't you just try the others that accept dist objects? $\endgroup$ – Anony-Mousse Dec 10 '18 at 6:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.