# Simple OLS with two samples

I want to obtain an unbiased estimator for $b_1$ in a simple regression like that: $Y_i = B_0 + B_1X_i + u_i$ when I have two samples, always same size for Y and X, but once the sample size is l and once the sample size is m. The respective sample means $\bar{Y_l},\bar{X_l}$ and $\bar{Y_m},\bar{X_m}$ are given. Now I wonder how I cans tart to get an unbiased estimator?

My idea was to use the 'normal/one-sample' formula and just put weights (correcting for different sample size between the two independent sets of data) in front.

An estimator for $b_1$ would be: (X'X)$^{-1}$X'Y without matrices: $\frac{\sum X_iY_i - N \bar{Y}\bar{X}}{\sum X_i^2 -N \bar{X}^2}$

which I wanted to modify to $\frac{l}{m+l} \frac{\sum X_iY_i - L \bar{Y_l}\bar{X_l}}{\sum X_i^2 - L \bar{X_l}^2} + \frac{m}{m+l} \frac{\sum X_iY_i - M \bar{Y_m}\bar{X_m}}{\sum X_i^2 - M \bar{X_m}^2}$

The capital M and L denoting the respective sample size.

Now I am not sure if my result is right, as I cannot show if it is unbiased, to be honest.

Is it unbiased in probabilistic terms? Or is it just a wrong estimator?

• Running a single multiple regression on all observations with X and an indicator for the sample as the two regressors will give you an unbiased estimator of the true linear effect of X (given the linear structure of the model is correct). – Michael M Oct 26 '13 at 11:08
• Thanks. I can assume the linear structure as shown above as correct. However, I don't see the point of using a dummy. From the "consistency of the sample mean" I have to assume with greater sample size both sets get the same means (in X and Y respectively) and therefor the dummies' significance will reach zero, or? – luigi martini Oct 26 '13 at 12:00

The unbiasedness poperty of the OLS estimator in the linear regression model is a finite-sample property, and it is based on a specific assumption of the model being correct -that the regressors are "strictly exogenous to the error term", namely $E(u_i|\mathbf X)=0$.

So if you accept that this assumption holds, as you indicate in a comment, and so the OLS estimator for each sample has the unbiasedness property, then a combination of the two will be unbiased if it is a linear combination with weights adding up to unity (but not necessarily a convex combination). Namely, let $\hat B_{1l}$ and $\hat B_{1m}$ be the two single sample estimators. Consider an estimator that it is some function of the two:

$$\hat B^* = h\left(\hat B_{1l},\hat B_{1m}\right)$$ Its expected value is

$$E\left[\hat B^*\right] = E\left[h\left(\hat B_{1l},\hat B_{1m}\right)\right]$$

If $h()$ is not an affine function, then by Jensen's inequality $$E\left[h\left(B_{1l},\hat B_{1m}\right)\right] \neq h\left(E\hat B_{1l},E\hat B_{1m}\right)$$ and in general $\hat B^*$ won't be unbiased.

Assume now that $h()$ is affine namely

$$\hat B^* = a_0 +a_1\hat B_{1l}+a_2\hat B_{1m}$$

with $a$'s being constants. Then

$$E\left[\hat B^*\right] = a_0 +a_1E\hat B_{1l}+a_2E\hat B_{1m} =a_0 + (a_1+a_2)B_{1}$$

For $$E\left[\hat B^*\right] = B_{1} \Rightarrow a_0 = (1-a_1-a_2)B_{1}$$

This condition depends on the unknown coefficient $B_1$ except if we set $a_0=0,\; a_1=1-a_2$, in which case it will hold always. In principle, these conditions do not exclude the possibility that $a_2 >1, a_1<0$, in which case we have no longer a convex combination. But interpreting negative weights is difficult (although in forecasting literature negative weights have been found to increase efficiency occasionally), so usually we take the convex combination, i.e. $0<a_1<1,\; 0<a_2<1, \; a_1+a_2=1$.

• Thanks a lot. But isn't that very similar to my solution, stated in the first part, where my estimator hat weights from the "SHARE" of observations? And would $\hat{B}= \frac{\bar{Y}_l -\bar{Y}_m}{\bar{X}_l - \bar{X}_m}$ be an as good unbiased estimator? – luigi martini Oct 26 '13 at 13:36
• a) My answer showed theoretically that you get unbiasedness only if you apply a linear combination (and not just a convex combination as was stated in your question) b) Have you tried to consider the expected value $E\hat{B}$ of the estimator you propose in your comment? – Alecos Papadopoulos Oct 26 '13 at 14:34
• okay, I need a linear, not a convex combination. Sorry, I must have confused something. Okay, thanks. The expected value I obtain is probably wrong, this is why I cannot solve the task. I get: $E(\bar{X_l})= E(\bar{X_m}) = \mu_X$ which makes the denominator 0. And this makes both fractions going to infinity, but as they are connected with a minus, it goes to zero in the end. Can this be true? – luigi martini Oct 26 '13 at 16:15
• No it is not because $E(\hat{B})=E\left( \frac{\bar{Y}_l -\bar{Y}_m}{\bar{X}_l - \bar{X}_m}\right) \neq \frac{E\bar{Y}_l -E\bar{Y}_m}{E\bar{X}_l - E\bar{X}_m}$ – Alecos Papadopoulos Oct 26 '13 at 18:25
• Oh - really not? Any hints how I can obtain the true expected value? – luigi martini Oct 27 '13 at 0:01