In linear regression I have come across a delightful result that if we fit the model
$$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$
then, if we standardize and centre the $Y$, $X_1$ and $X_2$ data,
$$R^2 = \mathrm{Cor}(Y,X_1) \beta_1 + \mathrm{Cor}(Y, X_2) \beta_2.$$
This feels to me like a 2 variable version of $R^2 = \mathrm{Cor}(Y,X)^2$ for $y=mx+c$ regression, which is pleasing.
But the only proof I know is not in anyway constructive or insightful (see below), and yet to look at it it feels like it should be readily understandable.
Example thoughts:
- The $\beta_1$ and $\beta_2$ parameters give us the 'proportion' of $X_1$ and $X_2$ in $Y$, and so we are taking respective proportions of their correlations...
- The $\beta$s are partial correlations, $R^2$ is the squared multiple correlation... correlations multiplied by partial correlations...
- If we orthogonalize first then the $\beta$s will be $\mathrm{Cov}/\mathrm{Var}$... does this result make some geometric sense?
None of these threads seem to lead anywhere for me. Can anyone provide a clear explanation of how to understand this result.
Unsatisfying Proof
\begin{equation} R^2 = \frac{SS_{reg}}{SS_{Tot}} = SS_{reg} = \langle(\beta_1 X_1 + \beta_2 X_2)^2\rangle \\= \langle\beta_1^2 X_1^2\rangle + \langle\beta_2^2 X_2^2\rangle + 2\langle\beta_1\beta_2X_1X_2\rangle \end{equation}\begin{equation} R^2 = \frac{SS_{reg}}{SS_{Tot}} = \frac{SS_{reg}}{N} = \langle(\beta_1 X_1 + \beta_2 X_2)^2\rangle \\= \langle\beta_1^2 X_1^2\rangle + \langle\beta_2^2 X_2^2\rangle + 2\langle\beta_1\beta_2X_1X_2\rangle \end{equation}
and
\begin{equation} \mathrm{Cor}(Y,X_1) \beta_1 + \mathrm{Cor}(Y, X_2) \beta_2 = \langle YX_1\rangle\beta_1 + \langle Y X_2\rangle \beta_2\\ =\langle \beta_1 X_1^2 + \beta_2 X_1 X_2\rangle \beta_1 + \langle \beta_1 X_1 X_2 + \beta_2 X_2^2\rangle \beta_2\\ =\langle \beta_1^2 X_1^2\rangle + \langle \beta_2^2 X_2^2 \rangle + 2\langle \beta_1 \beta_2 X_1 X_2\rangle \end{equation}
QED.
