When does R-squared in multiple linear regression equals the sum of the R-squared from two simple linear regression?

Question

I know that in the simple linear regression, the $R^2$ is just the sample correlation between the response and covariate. My question is that suppose I fit a linear regression by using two covariates, say $X_1$ and $X_2$, under which condition the $R^2$ in the multiple linear regression equals the sum of two $R^2$s from two simple linear regression?

My guess is that this only true when the sample correlation between $X_1$ and $X_2$ is zero, but I don't know how to show that. Also if we drop this assumption, what is largest and smallest possible values of $R^2$?

Answer 1

You can find a generalised explanation of this issue (for multiple linear regression), along with a general discussion of the geometric properties of linear regression, in O'Neill (2019). To facilitate analysis, let $R_1$ denote the sample correlation between $\mathbf{y}$ and $\mathbf{x}_1$, let $R_2$ denote the sample correlation between $\mathbf{y}$ and $\mathbf{x}_2$ and let $R_{1,2}$ denote the sample correlation between $\mathbf{x}_1$ and $\mathbf{x}_2$. From the result on p. 12 of the linked paper, you can write the coefficient of determination for the two-explanatory-variable multiple linear regression model as:

$$R^2 = \frac{1}{1-R_{1,2}^2} (R_1^2 + R_2^2 - 2 R_{1,2} R_1 R_2).$$

As you correctly predicted, if $\mathbf{x}_1$ and $\mathbf{x}_2$ are uncorrelated then we have $R_{1,2}=0$ which then gives:

$$R^2 = R_1^2 + R_2^2.$$

More generally, the coefficient of determination in a multiple linear regression can be written as a quadratic form determined entirely by the sample correlations between all the vectors in the data. The general form is shown and discussed in the linked paper (see also this related post). You might also be interested to note that the coefficient of determination for a multiple linear model can actually be larger than the sum of the coefficients of determination for each individual simple regression with the same data, meaning that the information in the multiple model is greater than the sum of its parts! (This phenomenon is called "enhancement" --- see pp. 14-16 of the linked paper.)