Suppose we have two orthogonal features $x_1$ and $x_2$. If we run univariate regression of $y$ on $x_1$ or $x_2$ and get $R^2, $$r_1$ and $r_2$ and we run multivariate regression of $y$ on both $x_1$ and $x_2$ and get the $R^2$ which is denoted by $r$. A well-known result is that $r = r_1 + r_2$. I am interested in how can we connect this result to the cosine angel. We know that the $cos\theta$ is the correlated between two vectors that have an angel $\theta$. An example is that if we represent $x_1 = (1, 0, 0)$ and $x_2 = (1, 0, 0)$. We further write $y = (a, b, 1)$. We know the regression $R^2$ is: $r_1 = \frac{a}{\sqrt{(a^2 + b^2 + 1)}}$ and $\frac{b}{\sqrt{(a^2 + b^2 + 1)}}$.
If we project $y$ on the space spanned by $x_1$ and $x_2$, we know the projection can be written as $z = (a, b, 0)$. The correlation between this projection and $y$ is $\frac{\sqrt{(a^2+b^2)}}{\sqrt{(a^2 + b^2 + 1)}}$ which is $r$. However it does not seem that this is the sum of $r_1$ and $r_2$. Did I miss anything?
