Recursive feature elimination (RFE) is a feature-selection strategy. It performs in two nested levels of cross-validation. First it tries to divide the training set into N folds. RFE puts one fold aside for testing the generalization and then trains itself with the remaining data.

In my case I only have two classes and would like to do RFE. The problem here is that, since there are only two classes, the algorithm can not put one aside and then train itself with the remaining folds (only one fold remains and training on single class is not sensible.)

I would be very much grateful if you could let me know what I could do here.

  • $\begingroup$ I need some clarification: When dividing data into 'folds', the number of classes is irrelevant, unless you're actually saying that you have only two data points. Why not randomly assign points to folds, without considering 'class'? (I assume that by 'class' you mean some binary response variable that is included in each point) $\endgroup$ – zkurtz Feb 4 '14 at 11:58
  • $\begingroup$ That is right. I assume I misunderstood the concept behind RFE. Aren't our classes the outcome that we are interested in? So what is the point of just using random folds? Just finding features with maximum variance? You can put your response below. Thanks. $\endgroup$ – Arman Feb 4 '14 at 14:53

When you say class, I hope you mean 'output class' (prediction value). As zkurtz said, number of output classes is irrelevant in terms of recursive feature elimination. So what it does is to select the max number of features that gives high precision and recall values for classification (regardless it's for 2 classes or more). And the whole part of dividing the training set into N folds means dividing the data points into N groups not dividing the features into N groups; and the point of dividing training set into N folds is for cross validation.

However if you mean 'feature', doesn't matter with RFE, it would only have 4 possible results. You can just get the results from calculating correlation scores between features and target vectors.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.