3
$\begingroup$

When applying a multiple linear regression, does the adjusted R-squared value depend on the number of independent variables in the model or the number of terms? Specifically, I'm concerned that adding interaction terms while keeping the number of independent variables the same may artificially inflate my adjusted R-squared value. Would this be the case? If so, is there a better metric of fit that I could use that acknowledges these additional interaction terms?

$\endgroup$
  • $\begingroup$ In multiple regression you typically only have one dependent variable. If you have more than one (in what you might call multivariate regression - but sometimes this term is used for multiple regression) you get a different R-squared for each dependent variable. If you add interaction terms, you're adding variables - a variable is a term (I think). $\endgroup$ – Jeremy Miles Feb 24 '15 at 22:03
  • 1
    $\begingroup$ Sorry, I meant independent throughout my post. I'll edit it accordingly. $\endgroup$ – Lyngbakr Feb 24 '15 at 22:07
3
$\begingroup$

The adjustment is for the number of terms in the regression

If you add interactions, adjusted $R^2$ is not "inflated" because of them ... if the terms add nothing of value, adjusted $R^2$ goes down just as it does when you add new variables that don't relate to the response.

See Wikipedia's article on Coefficient of determination, in the section on Adjusted $R^2$:

a modification due to Theil[7] of $R^2$ that adjusts for the number of explanatory terms in a model relative to the number of data points

... and then the formulas given indicate the same thing:

$$\bar R^2 = {1-(1-R^{2}){n-1 \over n-p-1}} = {R^{2}-(1-R^{2}){p \over n-p-1}}$$

(since interactions count in the count of parameters) and

$$\bar R^2 = {1-{SS_\text{res}/df_e \over SS_\text{tot}/df_t}}$$

(i.e. because it uses a term with the df for error, which will go down as you add interaction terms, it clearly adjusts for the effect of adding terms whether they're interaction terms or not).

So adjusted $R^2$ unambiguously accounts for the effect of adding new terms into your model, whether they're from interactions between existing variables or from additional variables.

$\endgroup$
  • 1
    $\begingroup$ Thanks for the reply and the helpful information, Glen. What confused me in the Wikipedia article and prompted me to ask the question was the phrase "total number of regressors" when defining p. When I looked regressor up also on Wikipedia, the definition given says it's synonymous with independent variable. The explanation you give clears this up. $\endgroup$ – Lyngbakr Feb 24 '15 at 23:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.