How can one quickly showthat the coefficient of determination, $\ R^2 $ , for a linear regression model containing an intercept is invariant with respect to linear transformations of the dependent variable, whereas the uncentered coefficient of determination is not?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ I believe you mean to write affine transformations instead of linear transformations. Strictly speaking, a linear transformation of a (univariate) response just multiplies its values by a constant. An affine transformation may also add a constant to all values. That distinction ought to give you a strong hint concerning what's going on: since such an additive constant (obviously) is a multiple of the intercept term, it will be completely accounted for by the intercept, leaving the fitted values unchanged--whence $R^2$ will not change, either. $\endgroup$– whuber ♦Commented Dec 1, 2016 at 17:51
-
1$\begingroup$ Bonsaibubble, after having read several of your posts I suggest you make your future titles more informative. This one could have been "Invariance of $R^2$ w.r.t. linear transformations" or the like. Your titles (like the current one) should not be like tags but should reflect the particular problem. $\endgroup$– Richard HardyCommented Jan 28, 2017 at 17:32
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Very quickly -- using the formula $$R^2 (centered) = \dfrac{Model\ Sum\ of\ Squares}{Model\ Sum\ of\ Squares - Residual\ Sum\ of\ Squares} = \dfrac{\sum(\hat y_i - \bar y)^2}{\sum(\hat y_i - \bar y)^2 - \sum(\epsilon_i)^2} $$
With a linear transformation of the dependent variable $y$, the distance between the fitted values and $\bar y$ remains the same. Likewise, since a linear transformation of the dependent variable changes the intercept, but not the slope, the residuals remain the same as well. So, $$MSS = MSS^\prime and\ RSS = RSS^\prime$$ where $MSS^\prime$ and $RSS$ are from the transformed linear model. Therefore, $$R^2(centered) = R^2(centered)^\prime$$
With the uncentered $R^2$ removes the mean of $y$ from the summation. $$R^2 (uncentered) = \dfrac{\sum(\hat y_i)^2}{\sum(\hat y_i)^2 - \sum(\epsilon_i)^2} $$ So, $$MSS \ne MSS^\prime and\ RSS = RSS^\prime$$ Therefore, $$R^2(uncentered) \ne R^2(uncentered)^\prime$$ The lack of centering for the uncentered $R^2$ means that the value changes after a linear transformation of $y$.
