# How to show unbiased estimator of combination of bernoulli and normal variables?

$X_1, X_2, \ldots, X_n$ is a random sample from $\mathrm{Bernoulli}(\theta)$, $\epsilon_1, \epsilon_2, \ldots, \epsilon_n$ are independent $\mathcal N(0, \sigma^2)$, independent of $X_i$.

Define $Y_i = \theta X_i + \epsilon_i$, for $i = 1, 2, \ldots,n$. Define estimating function $$\psi[\theta;(X,Y)] = \sum_{i =1}^n(Y_i - \theta X_i) \>.$$

a) Show that $\psi[\theta;(X,Y)]$ is unbiased estimator of $\theta$ if $\mathbb E(\psi[\theta;(X,Y)])=0$.

b) Find the estimator $\hat\theta$ such that $\psi[\hat\theta;(X,Y)] = 0$. Is $\hat\theta$ unbiased?

• Is this homework (it kind of reads like it)? If so, please add the homework tag. For such questions, we will provide hints, but not full solutions, in general. It is helpful if you also edit the question to include the work you've done and what specifically you are finding challenging. Feb 12, 2012 at 22:44
• Is this missing something? What exactly is the estimator? $\psi$ is a function of $\theta$, so it is not the estimator (as the question suggests). Is the estimator the minimizer of $\psi$? Feb 13, 2012 at 1:55
• @ Macro, Yes I missed some information. In a) there should be the condition "An estimator is unbaised if $E[\psi(\theta;(X,Y))] = 0$ and I got the solution using $E[Y] = \theta^2$ and $E[X]= \theta$. Now I want to know the idea of the question b). Feb 13, 2012 at 2:47
• Question a) certainly is absurd as stated: if $\mathbb{E}[\Psi(\theta;(X,Y))]=0$, $\Psi(\theta;(X,Y))$ cannot be unbiased. It can neither be an estimate since it depends on $\theta$! Feb 13, 2012 at 8:19
• @David: the question is not worded properly, it should state "is an unbiased estimator of $0$", not "is an unbiased estimator of $\theta$". Feb 13, 2012 at 13:24

I'm still not quite sure how part (a) is different from part (b) but, from your comment above it appears you are now only asking about part (b), so:

If $\psi[\hat{\theta};(X,Y)] = 0$, then

$$\sum_{i =1}^n(Y_i - \hat{\theta} X_i) \> = 0$$

So,

$$\sum_{i=1}^{n} Y_i = \hat{\theta} \cdot \sum_{i=1}^{n} X_i$$

Therefore $\hat{\theta} = \overline{Y}/\overline{X}$, the ratio of the sample means, satisfies $\psi[\hat{\theta};(X,Y)] = 0$. Regarding unbiasedness, it is easy to see that

$$E( \hat{\theta} ) = E\left( \frac{ \sum_{i=1}^{n} \theta X_i + \varepsilon_{i} }{ \sum_{i=1}^{n} X_i }\right) = \theta + E \left( \frac{ \sum_{i=1}^{n} \varepsilon_i }{ \sum_{i=1}^{n} X_i }\right),$$

Edit: Based on the discussion in the comments, I've edited my answer. Let $B=\sum_{i=1}^{n} X_{i}$ and $Z = \sum_{i=1}^{n} \varepsilon_i$. Then, $B \sim {\rm Binomial}(n,\theta)$ and $Z \sim N(0, n \sigma^{2})$. Since the errors are independent of the $X_{i}$,

$$E \left( \frac{ \sum_{i=1}^{n} \varepsilon_i }{ \sum_{i=1}^{n} X_i }\right) = E(Z) \cdot E \left( \frac{1}{B} \right)$$

Clearly $E(Z) = 0$. Assuming $\theta < 1$, $P(B = 0) = (1-\theta)^{n} > 0$. Therefore $E \left( \frac{1}{B} \right) = \infty$, so $E \left( \frac{ \sum_{i=1}^{n} \varepsilon_i }{ \sum_{i=1}^{n} X_i }\right)$ doesn't exist. Therefore $E(\hat{\theta})$ doesn't exist whenever $\theta < 1$, so $\hat{\theta}$ can't be unbiased (although it is consistent as long as $\theta > 0$).

• This seems to be a bit of a weird problem since $\mathbb P(\bar X_n = 0) >0$ for all $n$. I've only thought about this for a moment, but navigating around this fact looks difficult at the moment. Feb 13, 2012 at 14:44
• Good point, cardinal. I was thinking of this in the case where $X$ has an arbitrary distribution but didn't think of the case where there is mass at 0. So, of course, the logic above only applies when $E(1/\overline{X})$ exists. Feb 13, 2012 at 23:54
• Maybe the answer is that $\hat{\theta}$ is not unbiased because $E(\hat{\theta})$ doesn't exist? Feb 14, 2012 at 0:01
• I think that would be my answer, too. Honestly, it makes me suspicious that either (a) the OP hasn't quite stated the problem as intended or (b) the question was poorly thought out in the first place. Feb 14, 2012 at 0:52
• Thank you so much all. Its really good discussion. Finally, we got the estimator $\hat\theta$ is not unbaised. Feb 14, 2012 at 2:59