# How to compare variance explained in two regression models?

Let's say I have a measurement of height of 200 individuals as $$y_1$$ and fit a simple linear regression model as:

$$y_1 = x_1 + x_2 + \epsilon_1$$

where $$x_1$$ and $$x_2$$ are some subject metadata

I would use $$R^2$$ or $$R^2adjusted$$ to quantify the variance of $$x_1$$ and $$x_2$$ explained by $$y_1$$

Let's say I have a new measurement of height of the same 200 individuals (from a different measurement technique) as $$y_2$$ and a fit a simple linear regression model as:

$$y_2 = x_1 + x_2 + \epsilon_2$$

Same as above, I would use $$R^2$$ or $$R^2adjusted$$ to quantify the variance of $$x_1$$ and $$x_2$$ explained by $$y_2$$

Here, since $$x_1$$ and $$x_2$$ are subject metadata they are same in both models.

How should I compare the difference in variance explained by $$y_1$$ and $$y_2$$? Simply, I could use the difference in $$R^2$$ but I am not sure if this approach is correct and also could not think of a proper way to test the significance of the difference in $$R^2$$.

In other words, how would I pick a measurement method based on the variance (of $$x_1$$ and $$x_2$$) explained?

## 1 Answer

First, note that $$R^2$$ describes the proportion of variance in $$y$$ explained by the $$x$$ features.

Getting to your question, I would use an indicator variable $$m$$ (for “measurement technique”) to indicate which measurement technique was used. Then you wind up with $$400$$ observations and a regression like:

$$y=\beta_0+\beta_1x_1+\beta_2x_2+\beta_3m+\epsilon$$

Then you test if the $$m$$ variable is a significant contributor. This is the $$t$$-test of the coefficient on $$m$$, and this is common to have in software outputs.

With $$400$$ points, you might have enough observations to interact $$m$$ with the $$x$$ features, resulting in a regression like:

$$y=\beta_0 +\beta_1x_1 +\beta_2x_2 +\beta_3m +\beta_4x_1m +\beta_5x_2m +\epsilon$$

A reasonable test in this case would be a “chunk test” of the coefficients that involve $$m$$, the common one in this situation being a partial $$F$$-test of nested models with a null model $$y=\beta_0+\beta_1x_1+\beta_2x_2$$ nested within the above equation.

In fact, the $$t$$-test also can be seen as a chunk test, with the same null model nested within $$y=\beta_0+\beta_1x_1+\beta_2x_2+\beta_3m$$.

To measure the effect size, you have the $$R^2$$ (or adjusted $$R^2$$) from the model with $$m$$ (either alone or with interactions) and the model that excludes $$m$$. Consider their difference.

Throughout all of this, whether you model with $$m$$ or not, you use all $$400$$ observations.

In more generality than just your question, understanding chunk tests reveals a lot of beauty in statistics, as many common procedures are special cases of testing nested models.