I've a facebook users dataset in which each user has a "huge" set of attribute, i.e about 220 attributes like age, hometown, religion, and a set of facebook liked pages to store the users tastes.

Now I need an automatic tool (i've no time to develop an entire framework) to select only the relevant features in the dataset, in order to make agile the cluster algorithms that i've to use.

My only constraint is that I need a unsupervised features selection technique, since I've no further information than my initial dataset.

Any help?

  • $\begingroup$ Please say what you mean by "relevant" and "agile." What is your research question? $\endgroup$ – rolando2 Apr 29 '14 at 12:26
  • $\begingroup$ relevant = more important. eg a variable with standard deviation equal to zero it is - surely - not relevant. agile = in order to avoid dimensionality curse I need to select only relevant features otherwise the clustering algorithms returns bad results $\endgroup$ – enneppi Apr 29 '14 at 12:40

To select the important variables (features) in your dataset, you could perform variable clustering.

In variable clustering, the correlation between the variables is of interest (you want to cluster variables that are highly correlated with each other). This R package may provide more insight into how it works and will actually implement a method for you to use.

One way of performing variable clustering that may provide some intuition into your problem is to perform PCA on your data and then cluster variables that are grouped together by the PCA loadings. In a sense, this is what the methods in that package do. Intuitively, one could say that they are highly correlated with the same dimension of the PCA subspace and therefore with each other - thus they should be clustered. Note that before performing PCA and other similar procedures, one should standardize the variables.

In the comments below, you express that this approach may not be automatic enough for your specific situation. You claim that you need to run variable selection on multiple datasets and aggregate the findings. An alternative with a more automatic selection could be to use sparse PCA (PCA-based methods should probably be acceptable since the variable clustering also uses PCA). You will need to select a level of "sparsity" (how many variables to exclude). If you can run Matlab, the simplest thing would probably be to use the package by Karl Sjorstrand and run spca using soft thresholding (see example in the function) to select the level of sparsity automatically. Do this for all of your subsets and then aggregate the results in a logical way (one could be to exclude variables that are always excluded, or excluded in almost all datasets). If you do not have Matlab, you can run spca in R using the elasticnet package, though I do not think it will automatically determine the sparsity parameters. You may find another R package that does though.

  • $\begingroup$ the problem is that, in reality, I don't have only one dataset: I have a principal dataset from which - for a certain reason - I sample about 50 sub-datasest. With this background is unfeasible to perform a features selection "by hand", i.e: the useful R package you have cited, in a sense, obligate me to study case-by-case the results of the variables cluster algorithms. So what I would like to have it is a trivial automatic tool with the sub-dataset in input and the best relevant features in output $\endgroup$ – enneppi Apr 29 '14 at 13:18
  • $\begingroup$ Could you describe what you mean a bit more. Do you mean that you would have a problem because you would need to perform this variable clustering analysis on 50 subdatasets, and then aggregate the results to obtain a global variable selection? $\endgroup$ – Deathkill14 Apr 29 '14 at 13:47
  • $\begingroup$ No.The main problem of my project is clusterize facebook users according with their tastes(i.e their FB likes) and their profile infos. For a certain reason, I don't have only 1 dataset: I've 50datasets on which I perform the clustering algorithms.Since I've these different datasets I need to perform features selection on each of them.So I get the feeling that the R package is not useful 'cause I would to study its results case-by-case to understand the relevant features of each datasets.Since they are 50 I would like to have an "dogmatic" tool that tells me which feature are relevants. $\endgroup$ – enneppi Apr 29 '14 at 14:06
  • $\begingroup$ i'm sorry because maybe i didn't explain well the probleam: but i don't have to aggregate the findings. I must perform the attributes selection on 50 (or more) of datasets, so - using the first R package you suggested - it seems a not enough automatic way: in fact following your way i need to study 50 (or more) var clustering results by hand. But I don't have to aggregate the results. Anyway, now I take a look at matlab package. I'll let you know my feelings about it. Thanks @Deathkill14 $\endgroup$ – enneppi Apr 29 '14 at 14:52
  • $\begingroup$ @Deathkill14 Could you explain: 'then cluster variables that are grouped together by the PCA loadings' please? The loadings have the same dimensions as the variables right? Why would you cluster these variables by PCA loadings? $\endgroup$ – bigTree May 27 '14 at 14:49

You can refer to the paper by P. Mitra, C. A. Murthy and S. K. Pal, named "Unsupervised Feature Selection using Feature Similarity, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol .24, No. 4, pp 301-312, April 2002". The code is available at http://cse.iitkgp.ac.in/~pabitra/paper/fsfs.tar.gz. The paper can be found here.

  • $\begingroup$ have you ever tried this? how to deal with the parameter which specifies the number of feature to delete? $\endgroup$ – enneppi May 9 '14 at 9:23
  • 1
    $\begingroup$ I have tried it for experiments. It has good classification accuracy. The author has taken the number of features close to half of the original size of feature set. I have tried this for other data sets too. The authors have also given plots of entropy vs reduced feature set size for 3 datasets. Th entropy drops sharply at half the original feature set size for these 3 data sets used in the paper. $\endgroup$ – Curious May 9 '14 at 13:06
  • $\begingroup$ thanks Curi. and what about the entropy? have you matlab code to calculate the entropy like described in the paper? I've tried to search the entropy code in the paper source code but i have not find it. $\endgroup$ – enneppi May 9 '14 at 13:53
  • $\begingroup$ Sorry, I don't have the entropy code. I just have the code given on the mentioned site. $\endgroup$ – Curious May 9 '14 at 14:03
  • $\begingroup$ hi Curious. a question. Is it right: the higher is the entropy the lower is the feature subset quality? Or the higher is the entropy the HIGHER is the feature subset quality? $\endgroup$ – enneppi Jun 1 '14 at 22:37

I reply to me and to those of you who are interested, I tried a simple matlab package SPEC, for (spectral) feature selection. You can find the paper here and the matlab code here.

The algorithm doesn't select automatically the relevant features, but it weighs the features according to their consistency in the dataset (low weight means high relevance), see the paper for more details.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.