0
$\begingroup$

Let's say that I have an input X = Q + W + Z and an output Y. Is it better to find a unique predictor using (X,Y) or to find 3 predictors using (Q,Y), (W,Y), (Z,Y) and then do the mean of the prediction for these 3 predictors ? I am looking for some theorical reasons to explane if the 2 procedures are equivalent or if one is better than the other.

Thanks !

$\endgroup$
3
$\begingroup$

These procedures are not equivalent, and the first method is generally considered better, at least in the linear case.

Consider, e.g. the situation (in your notation) where one has (X = Q + W + Z, Y), that is, Y as a function of X. We may rewrite our estimate as (again, assuming linearity),

$$\hat{Y} = X\hat{\beta} = Q\hat{\beta_1} + W\hat{\beta_2} + Z\hat{\beta_3}$$

Here, the parameters $\beta_i$ are estimated simultaneously and under some general conditions optimally weighted - informally, all relevant information is used in these estimates. When you instead run the three partial regressions, you are requiring,

$$\hat{Y_q} = Q\hat{\beta_q},~\hat{Y_w} = W\hat{\beta_w},~\hat{Y_z} = Z\hat{\beta_z}$$

And finally, $\hat{Y} = \frac{1}{3} \left[ \hat{Y_q} + \hat{Y_w} + \hat{Y_z} \right]$. Here the weights need not be optimal: why would the mean be the 'best' linear combination of these estimates?

Generally, these results will then not be equivalent (i.a. because the second specification will, by definition, suffer from omitted variable bias), and the first method should be preferred to the second.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.