2
$\begingroup$

I am reading the book "grokking Machine Learning" by Luis G. Serrano and came across the following sentence:

"It seems that clustering and dimensionality reduction are nothing like each other, but, in reality, they are not so different. If we have a table full of data, each row corresponds to a data point, and each column corresponds to a feature. Therefore, we can use clustering to reduce the number of rows in our dataset and dimensionality reduction to reduce the number of columns."

I have doubts about the statement that clustering reduces the number of rows. It seems that clustering only groups data without reducing their column number. Am I mistaken?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Yes, you are right. Of course, one could just retain only one row from each cluster. If doing so makes sense depends on the task at hand. $\endgroup$
    – PhoemueX
    Commented Jun 20 at 20:50
  • 2
    $\begingroup$ Furthermore, if one were to replace each cluster by a linear combination of its elements (such as their barycenter, which is the arithmetic mean), that would be mathematically the transpose of the linear combinations occurring in PCA, but with an important restriction: all the nonzero coefficients in each linear combination would occur within a single cluster. If one were to enforce that in a dimensionality reduction algorithm it would probably be characterized as "clustering" of the variables. Thus, there is a mathematically superficial similarity. $\endgroup$
    – whuber ♦
    Commented Jun 22 at 11:53

2 Answers 2

Reset to default
2
$\begingroup$

When you perform dimensional reduction via, say, PCA, then the steps are roughly:

  1. Identify the principal components

  2. Express every variable in terms of a combination of the principal components

  3. Drop the least significant principal components

Similarly, when you perform clustering, you can do something like this:

  1. Identify the clusters

  2. Associate every record with a cluster

  3. Replace the record data with a frequency table of each cluster

So instead of having a million records, you instead have a hundred records, each of which represents a single cluster, along with a measure of how big that cluster is.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Yes, after clustering, some actions can be performed to reduce the number of records in the dataset, but this reduction is not the task of the clustering algorithm; it involves other algorithms with their own names. $\endgroup$
    – Leox
    Commented Jun 21 at 11:39
  • $\begingroup$ "Drop" has various meanings in statistical computing, ranging from delete or erase data from the dataset in memory through omit from a model (e.g. exclude a redundant covariate) to ignore for substantive, statistical or scientific purposes. Again, practices vary, but many of us might calculate all possible principal components but then focus reporting on just some of them, all the way down to selecting one as the "best" single summary or plotting two as the "best" reduction to that many dimensions. Here best deserves all the qualifying quotation marks you can give it. $\endgroup$
    – Nick Cox
    Commented Jun 25 at 4:08
2
$\begingroup$

One could make new data, where each row somehow represents a cluster, but that is not the purpose of cluster analysis, nor, AFAIK, is it the usual practice. I'd say the usual purpose of cluster analysis is to have new insights about how the observations "go together" in ways that are not easy to see just by looking.

How those insights are used depends on what data one is clustering and why. Cluster analysis is used in all sorts of fields.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.