Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The (training) data contains 1280 observations with 1415 features. The test set has additional 380 observations. The data is sparse, that is, many of the variables has many zeros and few positive values. a typical variable\feature distributes with zeros in about 90% of the observations and small tail of few positive values The response is dichotomous, i.e zero-one classification.

I have tried to fit:

  • L1,L2 and mixtures of both regularized binomial(logistic) models with cv.glmnet from glmnet package.
  • A simple glm() model with 100 first selected variables after selecting them with stepwise forward procedure step().

surprisingly the second fit (hardthresholded model) preformed better on the training and test sets, measuring to goodness of fit with auc (area under the curve in the ROC curve) and misclassification error rate. It even preformed better than a model with first 100 variables selected by L1 regulation.(any idea why?)

For this reason and because of some regulatory conditions in the industry I would like to continue preform variable selection similar to stepwise algorithm but faster.

Any suggestions?

share|improve this question

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.