This question already has an answer here:

I am using R/RStudio to code a regression (and to optimize the function) over 50+ different variables. For the optimization to work I need to fit a higher order function (I am not sure to what degree I will use).

My question is this: is there a way in R to choose the best variables?


marked as duplicate by whuber Apr 11 '14 at 14:16

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 1
    $\begingroup$ Many, many answers to this question are available by searching our site for the tags: regression model-selection $\endgroup$ – whuber Apr 11 '14 at 14:16

You're probably looking for the step() function that works fairly well for linear regression (not sure for a higher order function though). It does a stepwise search to find a model by AIC. You may refer to its documentation for further details.

p.s. There's a section about Selecting predictors on Forecasting: principles and practice book,which you probably would like to read.


Short answer: No.

Longer answer: If you have 50 potential variables and may consider some higher order function (you don't say which) then you have got a preposterously large number of potential models. Even with just 50 variables and a linear fit, you have $2^{50}$ models. If you allow interactions (even just 2-way), that number explodes.

A package such as leaps may help to find models for further work, but it isn't going to solve your problem for you. Why not?

  1. Each model has to be tested for the assumptions of OLS regression (although you may want to use some other form of regression with fewer assumptions).

  2. leaps uses a statistic such as Cp or adjr2 to select models. This ignores other reasons to include a variable.

  3. By testing many models, the results will be distorted. Some of this may be gotten around by using training and testing data sets.

  4. As far as I know, leaps does not deal with the hierarchy problem. (Others will correct me if I am wrong).

You're going to have to do some work before you start the modeling process.


Not the answer you're looking for? Browse other questions tagged or ask your own question.