I came across the adjusted $R^2$ for multivariate linear models, $R^{2}_{adjusted} = 1 - \frac{SSE / (n-p-1)}{SSTO / (n-1)}$, and I was curious what kinds of properties this satisfies. (Googling was not very helpful). I can tell that if we add additional predictors that don't help to explain the observations, then this quantity will strictly decrease, but are there other interesting properties this satisfies, or is just another somewhat arbitrary way of measuring fit?
Thanks.
It makes a lot of sense.
$R^2$ measures the ratio of residual variance (numerator) to the total variance (denominator), calculating each variance as if the observations were the whole population (literally applying the discrete $\mathbb E\left[\left(X-\mathbb E\left[X\right]\right)^2\right]$ formula to the residuals (numerator) and the pooled distribution of all $Y$ values (denominator)).
$$ R^2=1-\dfrac{ SSE/n }{ SSTotal/n }\\ SSE\text{: Sum of squared residuals (“errors”)}\\ SSTotal\text{: Total sum of squared deviations of } y\text{ values from }\bar y $$
However, these are biased estimates of the respective variances!
By dividing by $n-p-1$ in the numerator and $n-1$ in the denominator, we now have a ratio of unbiased estimates of the respective variances.
