1
$\begingroup$

How can one quickly showthat the coefficient of determination, $\ R^2 $ , for a linear regression model containing an intercept is invariant with respect to linear transformations of the dependent variable, whereas the uncentered coefficient of determination is not?

$\endgroup$
2
  • $\begingroup$ I believe you mean to write affine transformations instead of linear transformations. Strictly speaking, a linear transformation of a (univariate) response just multiplies its values by a constant. An affine transformation may also add a constant to all values. That distinction ought to give you a strong hint concerning what's going on: since such an additive constant (obviously) is a multiple of the intercept term, it will be completely accounted for by the intercept, leaving the fitted values unchanged--whence $R^2$ will not change, either. $\endgroup$
    – whuber
    Dec 1 '16 at 17:51
  • 1
    $\begingroup$ Bonsaibubble, after having read several of your posts I suggest you make your future titles more informative. This one could have been "Invariance of $R^2$ w.r.t. linear transformations" or the like. Your titles (like the current one) should not be like tags but should reflect the particular problem. $\endgroup$ Jan 28 '17 at 17:32
2
$\begingroup$

Very quickly -- using the formula $$R^2 (centered) = \dfrac{Model\ Sum\ of\ Squares}{Model\ Sum\ of\ Squares - Residual\ Sum\ of\ Squares} = \dfrac{\sum(\hat y_i - \bar y)^2}{\sum(\hat y_i - \bar y)^2 - \sum(\epsilon_i)^2} $$

With a linear transformation of the dependent variable $y$, the distance between the fitted values and $\bar y$ remains the same. Likewise, since a linear transformation of the dependent variable changes the intercept, but not the slope, the residuals remain the same as well. So, $$MSS = MSS^\prime and\ RSS = RSS^\prime$$ where $MSS^\prime$ and $RSS$ are from the transformed linear model. Therefore, $$R^2(centered) = R^2(centered)^\prime$$

With the uncentered $R^2$ removes the mean of $y$ from the summation. $$R^2 (uncentered) = \dfrac{\sum(\hat y_i)^2}{\sum(\hat y_i)^2 - \sum(\epsilon_i)^2} $$ So, $$MSS \ne MSS^\prime and\ RSS = RSS^\prime$$ Therefore, $$R^2(uncentered) \ne R^2(uncentered)^\prime$$ The lack of centering for the uncentered $R^2$ means that the value changes after a linear transformation of $y$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.