I have seen many examples of using semi-supervised learning to reduce the the number of features in a data set, but I am wondering if it is possible to somehow reduce features with purely unlabeled data.

Trivially, we can remove features that are the same value for every instance or that are very redundant in data, as these obviously cannot help us when generating a model. However, might there be some other, more non-trivial manners we may remove features from unlabeled data?

Intuition tells me that this is not possible as you can only know what a feature does/measure its worth if you can see its effect on data, but perhaps there was some caveat I have not considered. I have done some searching on here and online but all seem to include positive examples. (I have also found papers with abstracts that seem to indicate this might be possible, but the papers themselves cost $$$ which I would prefer not to spend).

Any answers/comments (even if they have links to papers) would be much appreciated, even if this is not know to be possible.

  • 5
    $\begingroup$ What is the aim of your analysis? Why you need to select the features? What for? $\endgroup$ – Tim Aug 15 '18 at 19:33
  • $\begingroup$ @Tim am doing a project that involves clustering and I have many features currently, which I would like to boil down as much as possible while still getting good results. $\endgroup$ – Alerra Aug 15 '18 at 19:36
  • $\begingroup$ I'm sure you've heard about autoencoders. Why don't they fit your needs? $\endgroup$ – Aksakal Aug 15 '18 at 19:39
  • 1
    $\begingroup$ You might want to consider subspace clustering algorithms, which are meant to work with high-dimensional data. $\endgroup$ – sebp Aug 15 '18 at 19:42
  • $\begingroup$ @Aksakal I have done some investigation on autoencoders, and while they do look very useful for my needs and I will probably incorporate them into my project in some way, I am looking for as many possible methods as I can so that way I can get the best model for my needs. Appreciate the input! $\endgroup$ – Alerra Aug 15 '18 at 20:17

Mixture models allow to perform feature selection and clustering, simultaneously.

You can use the R package VarSelLCM. A tutorial of this package is available at http://varsellcm.r-forge.r-project.org/


This isn't sufficiently well motivated. Feature selection should not be done blindly, and your thinking should not be passed off onto the computer. You need to consider what you are trying to do and why, and what your features mean and how they may relate to the analysis.

Consider the classic clustering example:

entity   type_of  location
whale    mammal   water
monkey   mammal   land
banana   fruit    land

How should these data be clustered? It depends on the kind of potential clusters you are interested in. Selecting type_of will yield taxonomic clusters; selecting location will yield geographic clusters. Both / either would be 'right' in some sense. You need to decide what sense you are interested in. This is done prior to looking at the data, and is ultimately similar to feature selection in supervised modeling. Similar concerns can be raised with the unsupervised analysis of feature correlations (e.g., PCA or factor analysis).

Having figured out what variables are relevant for what you want to do, and having conducted a clustering, it isn't difficult to see which variables play the strongest roles in determining the the clustering result by examining the separability on each variable, or in groups of variables (see, Mirkin, 1999).

  • B. Mirkin. Concept learning and feature selection based on square-error clustering. Machine Learning, 35:25–39, 1999.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.