1
$\begingroup$

What is the best way to determine the degree of contribution a variable is making by its addition to a regression model. Suppose I have following regression model for OutNumeric which is a continuous positive numeric variable.

lm(OutNumeric~Anumeric+Bfactor, data=mydf)

Anumeric is also a positive continuous variable, while Bfactor is a factor variable with only 2 levels. I want to determine the degree of improvement to the model by another positive continuous variable Cnumeric.

lm(OutNumeric~Anumeric+Bfactor+Cnumeric, data=mydf)

I could think of following simple options:

Check R^2 
Check AIC, BIC
Run predict on an independent set and see correlation.

However, my main aim is to determine if Cnumeric is an important factor in determination of OutNumeric, rather than prediction.

$\endgroup$
1
  • 2
    $\begingroup$ "Best" is really not defined here. All your suggestions make sense. Best for your purpose likely exists though. Can you provide more details of your goal/investigation? Some things you probably already know: If all you care about is having a highly determined model, then partial $R^2$ makes sense. Along the same lines, an F value greater than 1 for that predictor means it accounts for more error than a randomly chosen variable (there may be a better way to phrase that), and a significant F tells you it likely doesn't have 0 contribution. AIC of course balances predictiveness with complexity. $\endgroup$
    – le_andrew
    May 7, 2015 at 14:35

1 Answer 1

0
$\begingroup$

There are two thinks to consider, which are related to being important: does the addition of Cnumeric in the model lower the error of the model, is the contribution of Cnumeric unique?

In your case your models are nested. (See http://www.public.iastate.edu/~alicia/stat328/Multiple%20regression%20-%20nested%20models.pdf) The F-test for nested models actually uses the difference in squared error of the models compared to the number of extra parameters (1 in your case) in the larger model. If the test turns out positive, then the contribution of Cnumeric could be seen as important; importance is difficult to assess without a cost function though. Saving an extra life is quite important, saving a grain of rice less so, significant in statistical terms does not always means relevant.

Even if the addition of Cnumeric does not reduce the error significantly, this could be due to collinearity. Cnumeric in that case does not add more than the other two variables. A model with only Cnumeric might explain Outnumeric equally well as the other two variables together. Welcome to collinearity! Without any information regarding causation it is difficult to assess which variable is the most important. Alternatively, if all the predictors are uncorrelated, then the contribution of each predictor can be seen as unique. In this case it is more easy to say which predictor is the most 'important'.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.