How to combine regression models?

Question

Say I have three data sets of size $n$ each:

$y_1$ = heights of people from the US only

$y_2$ = heights of men from the whole world

$y_3$ = heights of women from the whole world

And I build a linear model for each with factors $x_i$, $i = 1,..., k$:

$\hat{y}_{j} = \beta_{0} + \beta_{1}x_{1} + \beta_{2}x_{2} + \epsilon_{j}$

with $\epsilon$ having the usual properties for OLS. And I may use a factor $x_i$ in more than one regression.

---

My question is: How could I combine the regressions such that I can obtain estimates for:

$y_{12}$ = height of men from the US only

$y_{13}$ = height of women from the US only

for which I do not have data

---

I thought of perhaps some sort of weighting:

$ \hat{y}_{12} = w_{1} \hat{y}_{1} + (1 - w_{1}) \hat{y}_{2}$

but then I wouldn't know what to use for $w_1$.

Answer 1

It's not clear whether you want estimates of height for each individual man and woman (more of a classification problem) or to characterize the distribution of heights of each sex. I will assume the latter. You also do not specify what additional information you are using in your model, so I will confine myself to addressing the case where you only have height data (and sex data, in the case of non-US citizens).

I recommend simply fitting a mixture of distributions to the height data from the US only, because the distributions of height in men and women are reasonably different. This would estimate the parameters of two distributions that when summed together best describe the variation in the data. The parameters of these distributions (mean and variance, since a Gaussian distribution should work fine) give you the information you are after. The R packages mixtools and mixdist let you do this; I'm sure there are many more as well.

This solution may seem odd, because it leaves out all the information you have from outside the US, where you have know the sex and height of each individual. But I think it is justified because:

1) We have a very strong prior expectation that men are on average taller than women. Wikipedia's List of average human height worldwide shows not even one country or region where women are taller than men. So the identity of the distribution with the greater mean height is not really in doubt.

2) Integrating more specific information from the non-US data will likely involve making the assumption that the covariance between sex and height is the same outside the US as inside. But this is not entirely true - the same Wikipedia list indicates that the ratio of male to female heights varies between approximately 1.04 and 1.13.

3) Your international data may be much more complicated to analyse because people in different countries have wide variation in height distributions as well. You may therefore need to consider modelling mixtures of mixtures of distributions. This may also be true in the US, but it is likely to be less of a problem than a dataset that includes the Dutch (mean height: 184 cms) and Indonesians (mean height: 158 cms). And those are country-level averages; subpopulations differ to an even degree.