I am using R/RStudio to code a regression (and to optimize the function) over 50+ different variables. For the optimization to work I need to fit a higher order function (I am not sure to what degree I will use).

My question is this: is there a way in R to choose the best variables?

  • 1
    $\begingroup$ Many, many answers to this question are available by searching our site for the tags: regression model-selection $\endgroup$
    – whuber
    Apr 11 '14 at 14:16

You're probably looking for the step() function that works fairly well for linear regression (not sure for a higher order function though). It does a stepwise search to find a model by AIC. You may refer to its documentation for further details.

p.s. There's a section about Selecting predictors on Forecasting: principles and practice book,which you probably would like to read.


Short answer: No.

Longer answer: If you have 50 potential variables and may consider some higher order function (you don't say which) then you have got a preposterously large number of potential models. Even with just 50 variables and a linear fit, you have $2^{50}$ models. If you allow interactions (even just 2-way), that number explodes.

A package such as leaps may help to find models for further work, but it isn't going to solve your problem for you. Why not?

  1. Each model has to be tested for the assumptions of OLS regression (although you may want to use some other form of regression with fewer assumptions).

  2. leaps uses a statistic such as Cp or adjr2 to select models. This ignores other reasons to include a variable.

  3. By testing many models, the results will be distorted. Some of this may be gotten around by using training and testing data sets.

  4. As far as I know, leaps does not deal with the hierarchy problem. (Others will correct me if I am wrong).

You're going to have to do some work before you start the modeling process.


This site is temporarily in read only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .