Say I have an i.i.d. sequence sequence $X_1,\ldots X_n \sim \text{Bernoulli}(p)$, and I am interested in estimating $p^2$. Let $T$ denote $\sum_{i=1}^n X_i$. It turns out that the mle $\bar X^2 = \frac{T^2}{n^2}$ is biased, but the following estimator is not: $$\frac{T(T-1)}{n(n-1)}\,.$$

Now, by other means I know that this estimator is the least variance unbiased estimator for $p^2$. (Briefly, the Bernoulli distribution is a member of the exponential family, and $T$ is complete sufficient statistic for $p$, and thus $\frac{T(T-1)}{n(n-1)}$ is the LVUE for its expected value, $p^2$.)

However, I tried to show this fact another way, by showing that the estimator is actually efficient. There is a corollary of the Rao-Cramer lower bound (I got this from Casella & Berger's Statistical Inference, p. 341):

Let $X_1,\ldots,X_n$ be iid $f(\mathbf{x}|\theta)$, where $f(\mathbf{x}|\theta)$ satisfies the conditions of the Cramer-Rao Theorem. Let $L(\theta|\mathbf{x})$ denote the likelihood function. If $W(\mathbf{x})$ is any unbiased estimator of $\tau(\theta)$, then $W(\mathbf{x})$ attains the Cramer-Rao Lower Bound iff $$a(\theta)[W(\mathbf{x})-\tau(\theta)] = \frac{\partial}{\partial\theta} \log L(\theta|\mathbf{x})$$ for some function $a(\theta)$.

Being a member of the exponential family, our Bernoulli random variable easily satisfies the conditions for the CRLB Theorem. However, when I try to apply this Corollary the following happens:

$$\log L(p|\mathbf{x}) = \sum_{i=1}^n x_i \log(p) + (1-x_i)\log(1-p) = T \log(p) + (n-T)\log(1-p)$$ $$\implies \frac{\partial}{\partial p} \log L(p|\mathbf{x}) = \frac{T-np}{p(1-p)}$$ So for the lower bound to hold, we need to have: $$a(p)\left[\frac{T(T-1)}{n(n-1)}-p^2\right] = \frac{T-np}{p(1-p)}$$

As far as I can tell, there is no function $a(p)$ that can make this hold as the left side is quadratic in $T$ but the left side only linear.

The only resolution to this I can see is that there does not exist an unbiased, efficient estimator for $p^2$. Is this the case?

  • 1
    $\begingroup$ I want to make sure that your definition of "LVUE" is the MVU: Minimum Variance Unbiased" estimator. If that's the case, I don't really understand your application of the Cramer Rao Theorem. There's no apriori reason why the LVUE should attain the Cramer Rao lower bound (hence making it efficient), especially when you have a finite sample. $\endgroup$ – Alex R. Apr 6 '16 at 22:25
  • $\begingroup$ Yeah, I have LVUE = least variance unbiased estimator. I realized after a lot of typing that there is no reason the estimator must be efficient (because $p^2$ is not the parameter of the exponential family distribution, but a function of the parameter). But I decided to post anyway to see if there was any problems in my reasoning, either about the fact that that estimator is indeed the LVUE/MVU estimator and or the fact that it isn't efficient. (Note that the Corollary to the CR theorem is still applicable because it is an "iff" statement.) $\endgroup$ – mb7744 Apr 7 '16 at 1:30
  • 1
    $\begingroup$ Actually, $$\dfrac{T(T-1)}{n(n-1)}=\mathbb{E}[X_1X_2|T]$$is a Rao-Blackwell estimate, which thus cannot be dominated. $\endgroup$ – Xi'an Apr 8 '16 at 19:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.