15
$\begingroup$

As far as I understand, $R^2$ explains how well the model predicts the observation. Adjusted $R^2$ is the one that takes into account more observations (or degrees of freedom). So, Adjusted $R^2$ predicts the model better? Then why is this less than $R^2$? It appears it should often be more.

$\endgroup$
30
$\begingroup$

$R^2$ shows the linear relationship between the independent variables and the dependent variable. It is defined as $1-\frac{SSE}{SSTO}$ which is the sum of squared errors divided by the total sum of squares. $SSTO = SSE + SSR$ which are the total error and total sum of the regression squares. As independent variables are added $SSR$ will continue to rise (and since $SSTO$ is fixed) $SSE$ will go down and $R^2$ will continually rise irrespective of how valuable the variables you added are.

The Adjusted $R^2$ is attempting to account for statistical shrinkage. Models with tons of predictors tend to perform better in sample than when tested out of sample. The adjusted $R^2$ "penalizes" you for adding the extra predictor variables that don't improve the existing model. It can be helpful in model selection. Adjusted $R^2$ will equal $R^2$ for one predictor variable. As you add variables, it will be smaller than $R^2$.

$\endgroup$
  • $\begingroup$ It is not clear, how the adjusted R square achieves the pointed properties. That is, what is the formula and how it cause the properties? $\endgroup$ – Alexey Voytenko Oct 5 '15 at 1:21
  • $\begingroup$ Adj R^2 = 1 - ((n -1)/(n - k -1))(1 - R^2) $\endgroup$ – mountainclimber Oct 13 '17 at 3:53
  • $\begingroup$ Where k = # of independent variables, n = # observations $\endgroup$ – mountainclimber Oct 13 '17 at 3:55
  • $\begingroup$ attempting to account for statistical shrinkage - perhaps for overfitting? $\endgroup$ – Richard Hardy Apr 24 '18 at 8:27
-1
$\begingroup$

R^2 explains the proportion of the variation in your dependent variable (Y) explained by your independent variables (X) for a linear regression model.

While adjusted R^2 says the proportion of the variation in your dependent variable (Y) explained by more than 1 independent variables (X) for a linear regression model.

$\endgroup$
  • 1
    $\begingroup$ The distinction you are making between "independent variables" and "more than 1 independent variables" is not clear. Also, quoting Andy from below, "You don't really add new information to what was provided before." $\endgroup$ – amoeba Feb 14 '15 at 23:10
-2
$\begingroup$

R-Squared increases even when you add variables which are not related to the dependent variable, but adjusted R-Squared take care of that as it decreases whenever you add variables that are not related to the dependent variable, thus after taking care it is likely to decrease.

$\endgroup$
  • 3
    $\begingroup$ Given that this question already has an accepted answer, this should be more of a comment. You don't really add new information to what was provided before. $\endgroup$ – Andy Feb 8 '14 at 17:44

protected by kjetil b halvorsen Sep 11 '17 at 10:34

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.