I want to obtain an unbiased estimator for $b_1$ in a simple regression like that: $Y_i = B_0 + B_1X_i + u_i$ when I have two samples, always same size for Y and X, but once the sample size is l and once the sample size is m. The respective sample means $\bar{Y_l},\bar{X_l}$ and $\bar{Y_m},\bar{X_m}$ are given. Now I wonder how I cans tart to get an unbiased estimator?

My idea was to use the 'normal/one-sample' formula and just put weights (correcting for different sample size between the two independent sets of data) in front.

An estimator for $b_1$ would be: (X'X)$^{-1}$X'Y without matrices: $\frac{\sum X_iY_i - N \bar{Y}\bar{X}}{\sum X_i^2 -N \bar{X}^2}$

which I wanted to modify to $\frac{l}{m+l} \frac{\sum X_iY_i - L \bar{Y_l}\bar{X_l}}{\sum X_i^2 - L \bar{X_l}^2} + \frac{m}{m+l} \frac{\sum X_iY_i - M \bar{Y_m}\bar{X_m}}{\sum X_i^2 - M \bar{X_m}^2}$

The capital M and L denoting the respective sample size.

Now I am not sure if my result is right, as I cannot show if it is unbiased, to be honest.

Is it unbiased in probabilistic terms? Or is it just a wrong estimator?

  • 2
    $\begingroup$ Running a single multiple regression on all observations with X and an indicator for the sample as the two regressors will give you an unbiased estimator of the true linear effect of X (given the linear structure of the model is correct). $\endgroup$
    – Michael M
    Oct 26 '13 at 11:08
  • $\begingroup$ Thanks. I can assume the linear structure as shown above as correct. However, I don't see the point of using a dummy. From the "consistency of the sample mean" I have to assume with greater sample size both sets get the same means (in X and Y respectively) and therefor the dummies' significance will reach zero, or? $\endgroup$ Oct 26 '13 at 12:00

The unbiasedness poperty of the OLS estimator in the linear regression model is a finite-sample property, and it is based on a specific assumption of the model being correct -that the regressors are "strictly exogenous to the error term", namely $E(u_i|\mathbf X)=0$.

So if you accept that this assumption holds, as you indicate in a comment, and so the OLS estimator for each sample has the unbiasedness property, then a combination of the two will be unbiased if it is a linear combination with weights adding up to unity (but not necessarily a convex combination). Namely, let $\hat B_{1l}$ and $\hat B_{1m}$ be the two single sample estimators. Consider an estimator that it is some function of the two:

$$\hat B^* = h\left(\hat B_{1l},\hat B_{1m}\right) $$ Its expected value is

$$E\left[\hat B^*\right] = E\left[h\left(\hat B_{1l},\hat B_{1m}\right)\right] $$

If $h()$ is not an affine function, then by Jensen's inequality $$E\left[h\left(B_{1l},\hat B_{1m}\right)\right] \neq h\left(E\hat B_{1l},E\hat B_{1m}\right)$$ and in general $\hat B^*$ won't be unbiased.

Assume now that $h()$ is affine namely

$$\hat B^* = a_0 +a_1\hat B_{1l}+a_2\hat B_{1m} $$

with $a$'s being constants. Then

$$E\left[\hat B^*\right] = a_0 +a_1E\hat B_{1l}+a_2E\hat B_{1m} =a_0 + (a_1+a_2)B_{1}$$

For $$E\left[\hat B^*\right] = B_{1} \Rightarrow a_0 = (1-a_1-a_2)B_{1} $$

This condition depends on the unknown coefficient $B_1$ except if we set $a_0=0,\; a_1=1-a_2$, in which case it will hold always. In principle, these conditions do not exclude the possibility that $a_2 >1, a_1<0$, in which case we have no longer a convex combination. But interpreting negative weights is difficult (although in forecasting literature negative weights have been found to increase efficiency occasionally), so usually we take the convex combination, i.e. $0<a_1<1,\; 0<a_2<1, \; a_1+a_2=1$.

  • $\begingroup$ Thanks a lot. But isn't that very similar to my solution, stated in the first part, where my estimator hat weights from the "SHARE" of observations? And would $\hat{B}= \frac{\bar{Y}_l -\bar{Y}_m}{\bar{X}_l - \bar{X}_m}$ be an as good unbiased estimator? $\endgroup$ Oct 26 '13 at 13:36
  • $\begingroup$ a) My answer showed theoretically that you get unbiasedness only if you apply a linear combination (and not just a convex combination as was stated in your question) b) Have you tried to consider the expected value $E\hat{B}$ of the estimator you propose in your comment? $\endgroup$ Oct 26 '13 at 14:34
  • $\begingroup$ okay, I need a linear, not a convex combination. Sorry, I must have confused something. Okay, thanks. The expected value I obtain is probably wrong, this is why I cannot solve the task. I get: $E(\bar{X_l})= E(\bar{X_m}) = \mu_X$ which makes the denominator 0. And this makes both fractions going to infinity, but as they are connected with a minus, it goes to zero in the end. Can this be true? $\endgroup$ Oct 26 '13 at 16:15
  • $\begingroup$ No it is not because $E(\hat{B})=E\left( \frac{\bar{Y}_l -\bar{Y}_m}{\bar{X}_l - \bar{X}_m}\right) \neq \frac{E\bar{Y}_l -E\bar{Y}_m}{E\bar{X}_l - E\bar{X}_m}$ $\endgroup$ Oct 26 '13 at 18:25
  • $\begingroup$ Oh - really not? Any hints how I can obtain the true expected value? $\endgroup$ Oct 27 '13 at 0:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.