1
$\begingroup$

If I have a dataset that is labeled with positive and negative examples, and I'd like to cluster (i.e. unsupervised) only the positive examples, does it make sense to reduce dimensionality using feature selection methods (which require data to be labeled, i.e. supervised) before performing clustering on the data?

For example: I have data about a variety of people that either buy gum or not. I want to know what the different types of gum buyers might look like, hence I only want to cluster the positive examples. However, since I have so many attributes for each person, I want to reduce the number of features I use before clustering them. Does it make sense to use this classification to select features if I am only clustering the positive examples? What pitfalls am I not seeing?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ I would suggest performing some feature reduction before running a clustering algorithm (such as k-means). If you have a lot of features, and some of them happen to be irrelevant or noisy, then the clustering algorithm will struggle to find a good solution. Removing highly correlated variables should also help. You can also try Principal Component Analysis to reduce dimensionality, and then run the clustering algorithm on a reduced space (using the top principal components). Keep in mind that the feature reduction/selection should be based on the positive cases. $\endgroup$
    – Vishal
    Commented Jan 22, 2020 at 22:43
  • $\begingroup$ Hi Vishal, thank you very much for your comment. Could you elaborate more about your last sentence: "Keep in mind that the feature reduction/selection should be based on the positive cases"? It does make intuitive sense to me, but I could also see how this kind of clustering of a positive class could benefit from it. It would help to have more reasoning behind it, and if you answer it, I can mark this question as closed. $\endgroup$
    – tborenst
    Commented Feb 2, 2020 at 21:35
  • $\begingroup$ I was under the assumption that you are only interested in what the gum buyers look like. If that is your objective, then you can simply use the positive cases (buyers) to perform both (1) feature reduction, and (2) clustering. However, if you are interested in looking at what factors are important in discriminating between buyers and non-buyers, then clustering won't help. You would have to either build a (binary) classification model or perform a discriminant analysis to help understand which features are helpful in differentiating a buyer from non-buyer. $\endgroup$
    – Vishal
    Commented Feb 4, 2020 at 17:33
  • $\begingroup$ Thank you, Vishal. This is very helpful, and answers my question. $\endgroup$
    – tborenst
    Commented Feb 5, 2020 at 18:17

1 Answer 1

Reset to default
1
$\begingroup$

You can perform dimensionality reduction in an unsupervised manner such as using PCA. You can then perform clustering in as many components as you need. Be cautious of the information gain for each PCA component. Since it drops drastically as you increase the number of components and you have added noise.

example: enter image description here

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you lordanis. This didn't answer the question I had necessarily, but found it very useful. $\endgroup$
    – tborenst
    Commented Feb 5, 2020 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.