1
$\begingroup$

I am studying linear regression lately and I notice this adjusted r-squared formula in a youtube video:

$$adj. R^2 = \frac{\frac{SSE}{n-k}}{\frac{SSTO}{n-1}}$$

While the formula that I know is this:

$$adj. R^2 = \frac{\frac{SSE}{n-k-1}}{\frac{SSTO}{n-1}}$$

I was about to comment in the yt vid that something is wrong but some people agree with it.

  1. How come that both formula seems to be applicable?
  2. Why does the other one has a n-k degrees of freedom while the other has n-k-1?

I'm so confused. I don't think of a way that you can equate the other formula to another formula.

$\endgroup$
6
  • $\begingroup$ There is a vote to close here; i suspect it is because the question is not yet up to standards. Perhaps, firstly define what you mean by $R$, $SSE$ and $SSTO$, I know of the first two, but not the third. $\endgroup$
    – asymptotic
    Commented Jun 9, 2021 at 11:26
  • $\begingroup$ Have you compared the setup that you know with that in the video? It's a difference of $k$ and $k+1$. $\endgroup$ Commented Jun 9, 2021 at 11:42
  • 2
    $\begingroup$ Does their $k$ include the intercept? Does yours? $\endgroup$
    – Dave
    Commented Jun 9, 2021 at 12:00
  • $\begingroup$ He did not say if he included the intercept. In the formula that I know (the one with n-k-1), the intercept is included. Is that the difference? If so, why would he not include the intercept? $\endgroup$
    – Acaro
    Commented Jun 9, 2021 at 12:02
  • $\begingroup$ If you have three variables plus an intercept, that makes four parameters to estimate. Is that $k=4$, or is that $k=3$ plus the intercept? Either way, we divide by four. $\endgroup$
    – Dave
    Commented Jun 9, 2021 at 12:31

2 Answers 2

1
$\begingroup$

In the adjusted R squared the numerator should be the unbiased estimator of $\sigma^2$, namely the SSE divided by the degrees of freedom of the residuals, that is, the sample size minus the number of regression coefficients. Usually, $k$ denotes the number of explanatory variables, and thus you add $1$ for the intercept $\beta_0$ (hence $n - (k+1)$). However, sometimes $k$ denotes the number of coefficients including the constant (hence $n-k$).

$\endgroup$
1
  • $\begingroup$ Ohh thank you that really helps me to understand it. $\endgroup$
    – Acaro
    Commented Jun 10, 2021 at 2:01
1
$\begingroup$

I now understand what I'm asking with the help of Dave and V.Vancak from math.stackexchange.com, but if my wording is wrong please correct me. So, the adjusted r-squared formula:

$$adj. R^2 = \frac{\frac{SSE}{n-k}}{\frac{SSTO}{n-1}}$$

Indicates that k includes the explanatory variables and the intercept term. Which implies that k = number of parameters.

While this formula:

$$adj. R^2 = \frac{\frac{SSE}{n-k-1}}{\frac{SSTO}{n-1}}$$

Indicates that k = explanatory variables only and the 1 is the intercept term.

All in all, these two formula are the same.

$\endgroup$
2
  • $\begingroup$ I would prefer to describe $k$ as the dimension of the feature space, but you have the gist. $\endgroup$
    – Dave
    Commented Jun 10, 2021 at 11:54
  • $\begingroup$ I rarely heard of that term as of now, but yeah I think that would be the appropriate term. Thanks man. $\endgroup$
    – Acaro
    Commented Jun 11, 2021 at 3:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.