1
$\begingroup$

I had thought a property of correlation is that it is invariant to scaling and offsets $\hat{y} = x\hat{\beta}_1 + \hat{\beta}_0$, so $\hat{y}$ is just a scaled and offset version of $x$.

So shouldn't $corr(y, \hat{y}) = corr(y, x)$? Where does the absolute value come from?

Improve this question
$\endgroup$
3
  • $\begingroup$ There's a derivation of $\text{Cor}(aX+b, Y) = \text{sgn}(a) \, \text{Cor}(X,Y)$ in the question The equivalence of sample correlation and R statistic for simple linear regression $\endgroup$
    – Silverfish
    Commented Apr 12, 2021 at 0:04
  • $\begingroup$ @Silverfish I was literally looking your answer in that post earlier today for a different reason (mainly paid attention to the vector space part) $\endgroup$
    – student010101
    Commented Apr 12, 2021 at 1:01
  • $\begingroup$ @Silverfish This is one of those cases where I blindly did $\frac{a}{\sqrt{a^2}} = 1$ instead of $sign(a)$. $\endgroup$
    – student010101
    Commented Apr 12, 2021 at 1:02

1 Answer 1

Reset to default
2
$\begingroup$

Correlation is invariant to offsets and positive scaling. Negative scaling flips the sign.

If the correlation between $y$ and $x$ is negative, then $\hat\beta_1<0$, so $$\mathrm{corr}[x\hat\beta_1,y]=-\mathrm{corr}[x,y]=\left|\mathrm{corr}[x,y]\right|$$

Improve this answer
$\endgroup$
6
  • $\begingroup$ Is it possible for the correlation between $\hat{y}$ and $y$ to be negative in cases where you don't include the intercept? I know that $R^2$ can become negative in this situation, but I'm not sure about $R$. $\endgroup$
    – student010101
    Commented Apr 11, 2021 at 22:22
  • 1
    $\begingroup$ Yes, quite possible. You hear more about it for $R^2$ because it's surprising that $R^2$ can be negative $\endgroup$
    – Thomas Lumley
    Commented Apr 11, 2021 at 23:30
  • $\begingroup$ @student010101 Of course some caution is merited, since in regressions without intercept, the coefficient of determination "R-squared" isn't the square of R! For your new Q, try the following R code: x <- c(0,1,2); y <- c(2,0,1); mod <- lm(y ~ 0 + x); summary(mod); yhat <- fitted(mod); cat("x:", x, "\ny:", y, "\nyhat:", yhat, "\ncor(x,y) =", cor(x,y), "\ncor(y,yhat) =", cor(y,yhat)); plot(x, y); points(x, yhat, col = "red"); abline(mod, col = "red") Here $\hat y=0.4x$ and correlations between $x$ and $y$, and $y$ and $\hat y$, are both -0.5. $\endgroup$
    – Silverfish
    Commented Apr 11, 2021 at 23:56
  • $\begingroup$ @student010101 The correlation between $y$ and $\hat y$ (fitted without an intercept term) will be negative if $x$ and $y$ have a negative correlation, but the regression slope is still positive so the sign-flipping doesn't occur. The graph in my example makes it clear why this can happen. Since this is slightly different question to the one you originally asked, you should consider asking it as a new question (though worth checking for duplicates first, in case it's been asked already) $\endgroup$
    – Silverfish
    Commented Apr 12, 2021 at 0:09
  • $\begingroup$ @Silverfish Just to clarify, when you do involve an intercept, R-squared is ALWAYS the square of R right? $\endgroup$
    – student010101
    Commented Apr 12, 2021 at 1:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.