# How to show unbiased estimator of combination of bernoulli and normal variables?

$X_1, X_2, \ldots, X_n$ is a random sample from $\mathrm{Bernoulli}(\theta)$, $\epsilon_1, \epsilon_2, \ldots, \epsilon_n$ are independent $\mathcal N(0, \sigma^2)$, independent of $X_i$.

Define $Y_i = \theta X_i + \epsilon_i$, for $i = 1, 2, \ldots,n$. Define estimating function $$\psi[\theta;(X,Y)] = \sum_{i =1}^n(Y_i - \theta X_i) \>.$$

a) Show that $\psi[\theta;(X,Y)]$ is unbiased estimator of $\theta$ if $\mathbb E(\psi[\theta;(X,Y)])=0$.

b) Find the estimator $\hat\theta$ such that $\psi[\hat\theta;(X,Y)] = 0$. Is $\hat\theta$ unbiased?

-
Is this homework (it kind of reads like it)? If so, please add the homework tag. For such questions, we will provide hints, but not full solutions, in general. It is helpful if you also edit the question to include the work you've done and what specifically you are finding challenging. –  cardinal Feb 12 '12 at 22:44
Is this missing something? What exactly is the estimator? $\psi$ is a function of $\theta$, so it is not the estimator (as the question suggests). Is the estimator the minimizer of $\psi$? –  Macro Feb 13 '12 at 1:55
@ Macro, Yes I missed some information. In a) there should be the condition "An estimator is unbaised if $E[\psi(\theta;(X,Y))] = 0$ and I got the solution using $E[Y] = \theta^2$ and $E[X]= \theta$. Now I want to know the idea of the question b). –  David Feb 13 '12 at 2:47
Question a) certainly is absurd as stated: if $\mathbb{E}[\Psi(\theta;(X,Y))]=0$, $\Psi(\theta;(X,Y))$ cannot be unbiased. It can neither be an estimate since it depends on $\theta$! –  Xi'an Feb 13 '12 at 8:19
@David: the question is not worded properly, it should state "is an unbiased estimator of $0$", not "is an unbiased estimator of $\theta$". –  Xi'an Feb 13 '12 at 13:24

I'm still not quite sure how part (a) is different from part (b) but, from your comment above it appears you are now only asking about part (b), so:

If $\psi[\hat{\theta};(X,Y)] = 0$, then

$$\sum_{i =1}^n(Y_i - \hat{\theta} X_i) \> = 0$$

So,

$$\sum_{i=1}^{n} Y_i = \hat{\theta} \cdot \sum_{i=1}^{n} X_i$$

Therefore $\hat{\theta} = \overline{Y}/\overline{X}$, the ratio of the sample means, satisfies $\psi[\hat{\theta};(X,Y)] = 0$. Regarding unbiasedness, it is easy to see that

$$E( \hat{\theta} ) = E\left( \frac{ \sum_{i=1}^{n} \theta X_i + \varepsilon_{i} }{ \sum_{i=1}^{n} X_i }\right) = \theta + E \left( \frac{ \sum_{i=1}^{n} \varepsilon_i }{ \sum_{i=1}^{n} X_i }\right),$$

Edit: Based on the discussion in the comments, I've edited my answer. Let $B=\sum_{i=1}^{n} X_{i}$ and $Z = \sum_{i=1}^{n} \varepsilon_i$. Then, $B \sim {\rm Binomial}(n,\theta)$ and $Z \sim N(0, n \sigma^{2})$. Since the errors are independent of the $X_{i}$,

$$E \left( \frac{ \sum_{i=1}^{n} \varepsilon_i }{ \sum_{i=1}^{n} X_i }\right) = E(Z) \cdot E \left( \frac{1}{B} \right)$$

Clearly $E(Z) = 0$. Assuming $\theta < 1$, $P(B = 0) = (1-\theta)^{n} > 0$. Therefore $E \left( \frac{1}{B} \right) = \infty$, so $E \left( \frac{ \sum_{i=1}^{n} \varepsilon_i }{ \sum_{i=1}^{n} X_i }\right)$ doesn't exist. Therefore $E(\hat{\theta})$ doesn't exist whenever $\theta < 1$, so $\hat{\theta}$ can't be unbiased (although it is consistent as long as $\theta > 0$).

-
This seems to be a bit of a weird problem since $\mathbb P(\bar X_n = 0) >0$ for all $n$. I've only thought about this for a moment, but navigating around this fact looks difficult at the moment. –  cardinal Feb 13 '12 at 14:44
Good point, cardinal. I was thinking of this in the case where $X$ has an arbitrary distribution but didn't think of the case where there is mass at 0. So, of course, the logic above only applies when $E(1/\overline{X})$ exists. –  Macro Feb 13 '12 at 23:54
Maybe the answer is that $\hat{\theta}$ is not unbiased because $E(\hat{\theta})$ doesn't exist? –  Macro Feb 14 '12 at 0:01
I think that would be my answer, too. Honestly, it makes me suspicious that either (a) the OP hasn't quite stated the problem as intended or (b) the question was poorly thought out in the first place. –  cardinal Feb 14 '12 at 0:52
Thank you so much all. Its really good discussion. Finally, we got the estimator $\hat\theta$ is not unbaised. –  David Feb 14 '12 at 2:59
show 1 more comment