Is the multiple correlation coefficient the correlation between $y$ and $\hat y$?

Question

Wikipedia states

[Multiple correlation] is the correlation between the variable's values and the best predictions that can be computed linearly from the predictive variables.

But lecture notes state that the square of multiple correlation between $y$ and a vector of variables $X$ is equal to

$$\frac {Y^TX(X^TX)^{-1}X^TY}{Y^TY}$$

Which is equal to $$\frac {Y^T \hat Y}{Y^T Y}$$

(Where I assume that the first column of $X$ contains only $1$’s, so that there is a constant term in the model). But if the multiple correlation is really equal to the correlation between $Y$ and $\hat Y$, then the its square should rather be equal to $\frac {(Y^T \hat Y)^2}{(Y^T Y)\cdot (\hat Y^T \hat Y)}$.

I don't see that these are the same. Is wikipedia wrong, are my lecture notes wrong, or are these equal?

Answer 1

So, if we define the $n$ $\times$ $k$ matrix of variables $X$ (respectively, number of observations and number of independent variables) as including the constant, let say $X$ = $($$1_n$ $X^*$$)$, then the $R^2$ (in this case called "centered $R^2$") coincides with the squared simple correlation between $Y$ and $\hat{Y}$.

Indeed, defining:

$M_{[1]}$ =$I_{n}$ $-$ $1_n$$($$1'_n$$1_n$$)^{-1}$$1'_n$

where $I_{n}$ is the $n$ $\times$ $n$ matrix with all 1 on its main diagonal and 0 elsewhere, while $1_n$ is the $n$ $\times$ $1$ vector of all 1, then we have that:

$R^2$ = $\hat{Y}$$'$$M_{[1]}$$\hat{Y}$ $/$ $Y'$$M_{[1]}$$Y$

while the squared simple correlation between $Y$ and $\hat{Y}$ is given by:

$($$r^2_{Y, \hat{Y}}$$)^{2}$ = $($$\hat{Y}$$'$$M_{[1]}$$Y$$)^2$ $/$ $\hat{Y}$$'$$M_{[1]}$$\hat{Y}$$Y'$$M_{[1]}$$Y$

The proof of this fact follows from the fact that $\hat{Y}$$'$$M_{[1]}$$Y$ = $\hat{Y}$$'$$M_{[1]}$$\hat{Y}$ which, in turn relies on the fact that $Y$ = $\hat{Y}$ $+$ $E$, where $E$ is the residual vector.

The problem with your equations is that the first one represents the so called "Uncentered $R^2$", that is the $R^2$ for regression in which the $1_n$ vector is $not$ included in $X$. Under a statistical point of view, this means that we are omitting the constant from the regression.

In this case, the equivalence between the $R^2$ and the squared simple correlation between $Y$ and $\hat{Y}$ does not hold, unless all the variables have $0$ sample mean.

Hope the explanation was clear.

Best Regards,

Niccolo'.