Comparison of two estimators based on mean squared error

Question

I'm refreshing my knowledge of statistics, and I'm stuck on a problem that asks for a comparison between 2 estimators (unfortunately, I can't remember the source of the problem, but I do know its intent is to show that unbiased estimators aren't necessarily the best).

$\underline{Problem}$:

Your colleague wishes to estimate the unknown success probability $p$ of a 2-sided (but possibly unfair) coin. To do so, he flips the coin and observes whether it is heads ($X_1 = 1$) or tails ($X_1 = 0$). If he observes heads, he proceeds to flip the coin an additional $k-1$ times for a total of $k$ flips, and he records the total number of heads obtained in the $k$ flips $\left(\sum_{i=1}^k x_i \right)$. If he observes tails, he does not flip the coin any more and just records the value of $X_1$

Define

$$N = X_1I(X_1=0) + \left(1+\sum_{i=2}^k X_i \right)I(X_1 =1)$$ where $I$ is the indicator function. Compare the mean squared error of two candidate estimators of $p$: (1) $X_1$ or (2) $\frac{N}{k}$ (i.e., the sample proportion).

$\underline{Solution}$:

It is relatively straightforward to see that $X_1$, while unbiased, isn't a very good estimator for $p$ because it will say $p=0$ or $p=1$. However, because it is unbiased, the mean squared error is simply the variance of $X_1$, which is $p(1-p)$ since it is a Bernoulli trial.

To calculate the mean squared error for the 2nd estimator, we need to figure out both its variance and bias. I claim that

\begin{eqnarray} \mathrm{var}\left( \frac{N}{k} \right) &=& \mathrm{var} \left( \mathbb{E}\left( \frac{N}{k} \big| X_1 \right) \right) + \mathbb{E} \left( \mathrm{var} \left( \frac{N}{k} \big| X_1 \right) \right) \\ &=& \frac{p^2}{k^2}(k-1)p(1-p) + \\& &\frac{p-p^2}{k^2} \left(1 + 2 (k-1)p + (k-1)p(1-p) + \left((k-1)p \right)^2 \right) \; \; \text{(*)} \end{eqnarray}

and that the expected value of $\frac{N}{k}$ is

$$\mathbb{E} \left( \frac{N}{k} \right) = \mathbb{E} \left( \mathbb{E} \left( \frac{N}{k} \big| X_1 \right) \right) = \frac{p}{k} \left(1+(k-1)p \right)$$ which means $\frac{N}{k}$ is a biased estimator, with the bias being

$$\mathrm{bias} \left( \frac{N}{k} \right) = \left[\frac{p}{k} (1+(k-1)p) - p \right] \; \; \text{(**)}$$

Therefore, the mean squared error of $\frac{N}{k}$ is the sum of (*) and (**) squared.

Does this seem reasonable? I would have expected the mean squared error of the 2nd estimator $\left( \frac{N}{k} \right)$ to be smaller than the mean squared error of the 1st estimator ($X_1$), thereby showing what I believe is the point of the exercise (i.e., sometimes biased estimators can be more efficient). However, based on what I've done, this doesn't seem to be the case, which makes me think I messed up somewhere (i.e., things don't seem to cancel out, etc. when I try to simplify (*) + (**)$^2$). Any guidance or insights would be appreciated.

$\textbf{Edit}$

Here is how I'm calculating the pieces of (*) in case someone can point on my error

\begin{eqnarray} \mathrm{var}_{X_1} \left( \mathbb{E}_{\frac{N}{k}|X_1}\left( \frac{N}{k} \big| X_1 \right) \right) &=& \mathrm{var}_{X_1} \left(\frac{1}{k}\mathbb{E}_{N | X_1} \left(N | X_1 \right) \right) \nonumber \\ &=& \mathrm{var}_{X_1} \left( \frac{1}{k} \left( 0 \times (1-p) + \left(1+\sum_{i=2}^k x_i\right) \times p \right) \right) \nonumber \\ &=& \frac{1}{k^2}\mathrm{var}_{X_1} \left( p + p\sum_{i=2}^k x_i \right) \nonumber \\ &=& \frac{p^2}{k^2}\mathrm{var}_{X_1} \left(\sum_{i=2}^k x_i \right) \nonumber \\ &=& \frac{p^2}{k^2}(k-1)p(1-p) \end{eqnarray}

\begin{eqnarray} \mathbb{E}_{X_1} \left( \mathrm{var}_{\frac{N}{k}|X_1} \left( \frac{N}{k} \big| X_1 \right) \right) &=& \mathbb{E}_{X_1} \left( \frac{1}{k^2}\mathrm{var}_{N|X_1} \left( N | X_1 \right) \right) \nonumber \\ &=& \mathbb{E}_{X_1} \left( \frac{1}{k^2} \left( \mathbb{E}(N^2 | X_1) - \left( \mathbb{E}(N | X_1 \right)^2 \right) \right) \nonumber \\ &=& \mathbb{E}_{X_1} \left( \frac{1}{k^2} \left( p \left(1+\sum_{i=2}^k x_i \right)^2 - p^2 \left(1 + \sum_{i=2}^k x_i\right)^2 \right) \right) \nonumber \\ &=& \mathbb{E}_{X_1} \left( \frac{1}{k^2} \left( p-p^2 \right) \left(1+\sum_{i=2}^k x_i \right)^2 \right) \nonumber \\ &=& \frac{p-p^2}{k^2}\mathbb{E}_{X_1} \left(1+2\sum_{i=2}^k x_i + \left( \sum_{i=2}^k x_i \right)^2 \right) \\ &=& \frac{p-p^2}{k^2} \left(\mathbb{E}_{X_1}(1) + 2 \mathbb{E}_{X_1} \left(\sum_{i=2}^k x_i \right) + \mathbb{E}_{X_1}\left( \sum_{i=2}^k x_i \right)^2 \right) \\ &=& \frac{p-p^2}{k^2} \left(1 + 2 (k-1)p + \mathrm{var}\left(\sum_{i=2}^k x_i \right) + \left( \mathbb{E}\left(\sum_{i=2}^k x_i \right) \right)^2 \right) \\ &=& \frac{p-p^2}{k^2} \left(1 + 2 (k-1)p + (k-1)p(1-p) + \left((k-1)p \right)^2 \right) \end{eqnarray}

Answer 1

There is most likely some mistake in your calculation of $Var\left(\frac{N}{k}\right)$. First, note that the term $X_1I(X_1=0)$ is always 0, so it does not need to be considered further. Then,

$\mathbb{E}\left(N\big|X_1\right)=\mathbb{E}\left((1+\sum_{i=2}^kX_i)I(X_1=1)\big|X_1\right)=I(X_1=1)\mathbb{E}\left(1+\sum_{i=2}^kX_i\right)=I(X_1=1)(1+(k-1)p);$

hence,

$Var\left(\mathbb{E}\left(N\big|X_1\right)\right)=Var\left(I(X_1=1)(1+(k-1)p)\right)=(1+(k-1)p)^2Var(I(X_1=1))=(1+(k-1)p)^2p(1-p).$

Because the first term is always 0, we do not have to consider it or the covariance in the derivation of $Var\left(N\big|X_1\right)$. We get

$Var\left(N\big|X_1\right)=Var\left((1+\sum_{i=2}^kX_i)I(X_1=1)\big|X_1\right)=(I(X_1=1))^2 Var\left(1+\sum_{i=2}^kX_i\right)=(I(X_1=1))^2(k-1)p(1-p)$

and

$\mathbb{E}\left(Var\left(N\big|X_1\right)\right)=\mathbb{E}\left((I(X_1=1))^2(k-1)p(1-p)\right)=(k-1)p(1-p)\mathbb{E}\left((I(X_1=1))^2\right)=(k-1)p^2(1-p).$

Finally, we obtain

$Var\left(\mathbb{E}\left(\frac{N}{k}\big|X_1\right)\right)=\frac{1}{k^2}(1+(k-1)p)^2p(1-p)$;

$\mathbb{E}\left(Var\left(\frac{N}{k}\big|X_1\right)\right)=\frac{1}{k^2}(k-1)p^2(1-p)$;

MSE$\left(\frac{N}{k}\right)=\frac{p(1-p)}{k^2}(1+(k+2)(k-1)p)$ (after simplification).

This yields the following for $k=100$: