# Regression summaries in regression analysis

Why R-square and adjusted R-square are not close each other in many cases in regression summaries table in regression analysis?

-

Adjusted $R^2$ is one method of penalizing $R^2$ for the complexity of the model.

Specifically,

$\bar{R^2} = R^2 - (1-R^2)\frac{p}{n-p-1}$

where n is sample size and p is number of predictors.

So, if n is small relative to the number of predictors, there will be a big difference between the two measures.

-

$R^2$ is calculated as $$R^2 = \frac{RSS}{TSS}$$ so it's the residual sum of squares (RSS) divided by the total sum of squares (TSS). This measure is increasing in the number of regressors included in the model, so the more variables you add this will increase your $R^2$ (or at least it will not decrease it).

The adjusted version $\bar{R}^2$ takes this issue into account by calculating $$\bar{R}^2 = 1 - \frac{\frac{RSS}{N-K}}{\frac{TSS}{N-1}}$$ where $N$ is the number of observations and $K$ is the number of explanatory variables including the constant. From this you can see that the $\bar{R}^2$ penalizes the number of regressors in the model relative to their explanatory power. In this case $\bar{R}^2$ is allowed to decrease when you add another variable to the model. Note that $\bar{R}^2$ is always less than or equal to $R^2$.

When you have a larger difference between these two measures this is indicative for having too many variables in the regression that do not help much in explaining the dependent variable.

-
Two very different looking formulas for the same thing. Yours from ANOVA type reasoning, mine from regression. – Peter Flom Dec 15 '13 at 13:28