Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Why R-square and adjusted R-square are not close each other in many cases in regression summaries table in regression analysis?

share|improve this question

2 Answers 2

Adjusted $R^2$ is one method of penalizing $R^2$ for the complexity of the model.

Specifically,

$\bar{R^2} = R^2 - (1-R^2)\frac{p}{n-p-1}$

where n is sample size and p is number of predictors.

So, if n is small relative to the number of predictors, there will be a big difference between the two measures.

share|improve this answer

$R^2$ is calculated as $$R^2 = \frac{RSS}{TSS}$$ so it's the residual sum of squares (RSS) divided by the total sum of squares (TSS). This measure is increasing in the number of regressors included in the model, so the more variables you add this will increase your $R^2$ (or at least it will not decrease it).

The adjusted version $\bar{R}^2$ takes this issue into account by calculating $$\bar{R}^2 = 1 - \frac{\frac{RSS}{N-K}}{\frac{TSS}{N-1}}$$ where $N$ is the number of observations and $K$ is the number of explanatory variables including the constant. From this you can see that the $\bar{R}^2$ penalizes the number of regressors in the model relative to their explanatory power. In this case $\bar{R}^2$ is allowed to decrease when you add another variable to the model. Note that $\bar{R}^2$ is always less than or equal to $R^2$.

When you have a larger difference between these two measures this is indicative for having too many variables in the regression that do not help much in explaining the dependent variable.

share|improve this answer
3  
Two very different looking formulas for the same thing. Yours from ANOVA type reasoning, mine from regression. –  Peter Flom Dec 15 '13 at 13:28

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.