2
$\begingroup$

Here is my situation: - A huge amount of data - 600 features - Only one class is provided Now, my question is how can I reduce the number of features to important ones? In another word, all of these features (with data) are intending to predict only one class. but some of features have large impact on the prediction (means their variation come to higher probability).

$\endgroup$
  • $\begingroup$ There are a lot of selection criteria that only apply to certain estimation procedures/statistical models - what are you going to run on the data? $\endgroup$ – Jeremias K Jan 25 '16 at 8:46
2
$\begingroup$

There is no universal answer regarding feature selection. Most of the time, they also depend on the learning method you use.

There are model free methods, refered to as filters. that allow you to select predictors. Per example, ranking them with respect to the correlation with the output and retaining the variables with the highest correlation (in modulus), the highest mutual information... See http://www.jmlr.org/papers/volume3/guyon03a/guyon03a.pdf, for an introduction and examples.

The "bad thing" (all feature selection methods have drawbacks) is that they an miss interactions (two features on their own may be poor predictors, but their product may happen to be a good one). This article http://www.public.asu.edu/~huanliu/papers/ijcai07.pdf tackles this issue.

There are model specific methods Random forest, gradient boosting, per example, propose variable importance estimates. After running a random forest, you can use the importance produced to train a new random forest on the n% of the most important variables (n=70 is an example, you should cross-validate to find the best percentage to keep). These are called wrappers.

And you have embedded methods that perform both training and variable selection in the same time. As goodepic states it, LASSO and Elastic net allows you to select variables in the case of a linear model. These are a little bit specific as variable selection is performed during the training step.

At last, there is domain knowledge. Which corresponds to your prior belief about the relevancy of the predictors with respect to your target.

$\endgroup$
0
$\begingroup$

For feature selection (I assume you have lots of samples of your class) - You could simply use PCA, (which finds the features with the most variance) or other features such as UFF (Unsupervised feature filtering) , I don't know if other methods (matlab & Mutual information, or mrmr) will work with just one class.

For prediction - You might want to try using one-class SVM maybe? Or Positive-Unlabelled learning.

Do you want to predict? Cluster? Find information about the specific features that characterize a class?

$\endgroup$
0
$\begingroup$

There are tons of models for variable selection from the regression literature. Most have been extended to generalized linear models (which includes logistic/binomial regression for classification).

Consider the lasso, adaptive lasso, elastic net (does better with colinearity, which may be useful to you with 600 variables). If you have structure in your features, you can consider group lasso and the myriad developments from that under the term structured sparsity. Eric Xing's group at CMU has developed good work in that vein, and some very fast algorithms.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.