15
$\begingroup$

As far as I understand, $R^2$ explains how well the model predicts the observation. Adjusted $R^2$ is the one that takes into account more observations (or degrees of freedom). So, Adjusted $R^2$ predicts the model better? Then why is this less than $R^2$? It appears it should often be more.

$\endgroup$
30
$\begingroup$

$R^2$ shows the linear relationship between the independent variables and the dependent variable. It is defined as $1-\frac{SSE}{SSTO}$ which is the sum of squared errors divided by the total sum of squares. $SSTO = SSE + SSR$ which are the total error and total sum of the regression squares. As independent variables are added $SSR$ will continue to rise (and since $SSTO$ is fixed) $SSE$ will go down and $R^2$ will continually rise irrespective of how valuable the variables you added are.

The Adjusted $R^2$ is attempting to account for statistical shrinkage. Models with tons of predictors tend to perform better in sample than when tested out of sample. The adjusted $R^2$ "penalizes" you for adding the extra predictor variables that don't improve the existing model. It can be helpful in model selection. Adjusted $R^2$ will equal $R^2$ for one predictor variable. As you add variables, it will be smaller than $R^2$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ It is not clear, how the adjusted R square achieves the pointed properties. That is, what is the formula and how it cause the properties? $\endgroup$ – Alexey Voytenko Oct 5 '15 at 1:21
  • $\begingroup$ Adj R^2 = 1 - ((n -1)/(n - k -1))(1 - R^2) $\endgroup$ – mountainclimber11 Oct 13 '17 at 3:53
  • $\begingroup$ Where k = # of independent variables, n = # observations $\endgroup$ – mountainclimber11 Oct 13 '17 at 3:55
  • $\begingroup$ attempting to account for statistical shrinkage - perhaps for overfitting? $\endgroup$ – Richard Hardy Apr 24 '18 at 8:27
-1
$\begingroup$

R^2 explains the proportion of the variation in your dependent variable (Y) explained by your independent variables (X) for a linear regression model.

While adjusted R^2 says the proportion of the variation in your dependent variable (Y) explained by more than 1 independent variables (X) for a linear regression model.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ The distinction you are making between "independent variables" and "more than 1 independent variables" is not clear. Also, quoting Andy from below, "You don't really add new information to what was provided before." $\endgroup$ – amoeba Feb 14 '15 at 23:10
-2
$\begingroup$

R-Squared increases even when you add variables which are not related to the dependent variable, but adjusted R-Squared take care of that as it decreases whenever you add variables that are not related to the dependent variable, thus after taking care it is likely to decrease.

| cite | improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ Given that this question already has an accepted answer, this should be more of a comment. You don't really add new information to what was provided before. $\endgroup$ – Andy Feb 8 '14 at 17:44

Not the answer you're looking for? Browse other questions tagged or ask your own question.