2
$\begingroup$

I'm currently using R-Squared and adjusted R-Squared to determine goodness of fit of a linear model. Although I understand the difference between the two, I expect that for one predictor, both the R-Squared and the adjusted R-Square should have the same value as specified here. However, by looking in the equation of the adjusted R-Squared:

Adj_R2 = (1-{(1-R2)(n-1)}/(n-p-1)

for one predictor, p=1, and Adj_R2 would NOT be equal to R2. Can someone clarify this please? Is the equation correct?

$\endgroup$
1
$\begingroup$

In your definition, I'd suspect p is the number of regressors not including the constant. When doing linear regression analysis with matrix notation, you include a column of ones in your $X$ dependent variable matrix (for the intercept), so the definition of the adjusted R^2 (defined as $\bar{R}^2$) in the matrix case is $$\bar{R}^2 = 1-(1-R^2)\frac{n-1}{n-K}$$ where $K$ is the number of regressors in your dependent matrix.

As you can see, the matrix notation achieves equality between the $R^2$s when K = 1, and your notation achieves equality when p = 0, both of which are equivalent to stating that equality is achieved when you only include a constant coefficient, ie your model is $Y = \beta_0$. Once you include a single non constant regressor, you will get different results, with the adjusted R^2 being lower. Indeed, just try it in R!

x <- c(1,2,3,4,5,6)
y <- c(1,3,4,7,6,9)

summary(lm(y~x))$r.squared #.919
summary(lm(y~x))$adj.r.squared #.899

This is also a great time to again highlight how using R^2 and adjusted R^2 should only be used to evaluate how well the data fits the linear relationship specified in the model, and nothing about the quality of your model's regressors, causality, or anything else. You could have a model with a very small R squared that still has extremely important insights, and you could have a model with a very high R squared that is full of regressors that are useless and unrelated to the underlying model... I won't go on, but do be careful about using them (I especially find that interpretation of adjusted R^2, while good for correcting the problem of adding regressors, has the flaw that is has no lower bound and can range from $(-\infty,1]$, while R^2 ranges from $[0,1]$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.