1
$\begingroup$

Can I use predicted outcomes from one model as the dependent variable in another model to make causal claims? Put differently, is there something equivalent to the Frish-Waugh-Lovell theorem for working with predicted outcomes rather than residuals?

To explain and make things tangible, let's take something like the Boston housing data as an example and assume prices are actually only driven by the number of bathrooms and distance from nearest metro station, and I want to know the causal effect of an extra bathroom on prices, so the regression

$$price = \beta_0 + \beta_1 n\_bathrooms + \beta_2 distance$$

correctly identifies the causal effect $\beta_1$.

Now we know from FWL that running the regression on distance only first, and then using the residuals from that as the dependent variable in a univariate regression on $n\_bathrooms$ will recover the coefficient for $\beta_1$ from the bivariate regression.

Now let's assume I know that the linear model with $n\_bathrooms$ and $distance$ is the correct one, but I don't actually have access to distance data, only to predicted outcomes given a fixed distance. That is, someone else has run the "univariate" regression on distance already and has provided me with the predicted house prices for each house given a distance of 1 mile from the metro station. Now I want to run the regression

$$\hat{y} = \beta_0 + \beta_1 n\_bathrooms$$

and my question is: what can or can't I guarantee about the estimate $\beta_1$ given my assumptions?

I know the 2017 Mullainathan Spiess paper where they argue that one of the good use cases for ML methods in economics is 2SLS, where the first step is pure prediction and the estimator improves with the predictive power of the first step. It also seems to me conceptually that if I assume that the conditional independence assumption holds given distance and bathrooms, then I should be able to recover a causal effect given data on bathrooms and a distance-adjusted outcome measure (i.e. $\hat{y}$ in my example).

Some simple simulation studies I tried suggest that in simple cases the coefficient from the full regression is recoverable albeit not exactly (as there isn't a direct algebraic correspondence between the two like with the full and partial models in FWL), and I assume in general it depends on what the true model is, how well the partial model approximates it, whether there are interaction effects etc.

I haven't been able to find any references discussing this question, so any pointers would be appreciated.

$\endgroup$

1 Answer 1

1
$\begingroup$

I don't know any formal proofs, but it was fairly straightforward to disprove this idea with a simulation - see Stata code below. You can tweak the values for relationships between dist-> bath, and dist->y without altering bath->y, and you will see fairly dramatic changes.

clear
set obs 10000 

gen dist = rnormal()
gen bath = .1*dist + rnormal()
gen y = .5*dist + .25*bath + rnormal()

reg y bath dist

reg y dist 
predict yhat, xb

reg yhat bath
$\endgroup$
2
  • $\begingroup$ Thanks for your answer! If I understand your code correctly (I haven't used Stata in ages!) the issue comes from the correlation between dist and bath which you introduce when creating bath, correct? $\endgroup$
    – Nils Gudat
    Apr 26, 2023 at 10:13
  • $\begingroup$ Intuitively, the only way that y is associated with yhat|dist is through the backdoor path bath <- dist -> y. i.e. the association between bath and yhat|dist reflects the extent to which dist confounds the causal effect of bath->y. I can't see that there's any information about the causal effect of bath->y contained there. Does that help? $\endgroup$
    – MartinQLD
    Apr 27, 2023 at 3:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.