2
$\begingroup$

Let me state the theorem first:

Let $T$ be a sufficient and complete statistic for the statistical model $\mathcal{P}$ and let $\tilde{\gamma}_1$ be an unbiased estimator for the parameter $\gamma = g(\theta) \in \mathbb{R}^k$ then the estimator \begin{equation}\hat{\gamma} = \hat{\gamma}(T)=E.(\tilde{\gamma}_1 \mid T) \end{equation}

has the smallest covariance matrix among all unbiased estimators for the parameter $\gamma = g(\theta)$

Here is the example: let $\boldsymbol{X}$ be a sample of independent $N(\mu,\sigma^2)$ distributed r.v's with parameter if interest $\theta = (\mu,\sigma^2)$

The arithmetic mean $\bar{X} = \dfrac{1}{n}\sum X_i $ and sample variance $S^2 = \dfrac{1}{n-1} \sum (X_i - \bar{X})^2$ are unbiased estimators.

We also know that ($\sum X_i, \sum X^2_i$) is sufficient and complete for the class of all normal distributions.

....................

Iam trying to apply the theorem above therefore i first set $\tilde{\gamma}_1 = (\bar{X},S^2)^T$ and $T = (\sum X_i, \sum X^2_i)^T$

but does $\hat{\gamma}(T)=E.(\tilde{\gamma}_1 \mid T)$ mean: $E\begin{pmatrix} \bar{X} \mid \sum X_i \\S^2 \mid \sum X^2_i \end{pmatrix}$ ? this cannot be the right , can someone tell me what Iam doing wrong?


Let me take another example: Let $(X_1,...,X_n)$ be iid acording to the uniform distribution over the interval $[0,\theta]$ then $\hat{\theta}_{MLE}= X_{max}$ is an unbiased estimator and $E_{\theta}X_{max} = \dfrac{n}{n+1}\theta$.

We know that $X_{max}$ is complete and sufficient , hence according to the theorem

$E[\frac{n+1}{n}X_{max} \mid X_{max} ] = \theta $ but that is just the parameter i should get an estimator, i think I am missing something fundamental

$\endgroup$
  • 1
    $\begingroup$ I believe the Lehmann-Scheffe theorem requires completeness too and not just sufficiency. $\endgroup$ – dsaxton Jul 30 '15 at 20:44
2
$\begingroup$

It's actually easier than you're making it. Another way you can think about Lehmann Scheffe (or really Rao-Blackwellizing as that's what you're doing here) is that you're looking for a function of the sufficient statistics that is an unbiased estimator of whatever parameter you're estimating. The important bit there is recognizing that conditioning your estimator on the sufficient statistic accomplishes the same goal as finding a function of the sufficient statistics. That means that if the thing you start with as your unbiased estimator is already a function of the sufficient statistics, you're done. You've found the UMVUE.

In the context of the normal distribution both $\bar{X}$ and $S^{2}$ are already function of the UMVUE parameter (this is where your logic was wrong. You don't condition each separately but rather on the joint statistic $\vec{T(X)}=(\sum_{i}X_{i},\sum_{i}X_{i}^{2}$ ) and so they are already the UMVUE.

As far as the uniform family goes, you're computing the expectation wrong. $E[Y|Y]=Y$

$\endgroup$
  • $\begingroup$ ok thanks @Nick !!, but just for the sake of it $ \hat{\gamma}(T)=E.(\tilde{\gamma}_1 \mid T)$ means , in the case with the normal distribution example: $E[\bar{X} \mid \sum X_i , \sum X^2_i]$ and $E[S^2 \mid \sum X_i , \sum X^2_i]$ , $\endgroup$ – Danny Jul 30 '15 at 14:37
  • $\begingroup$ Really, you're condition the joint estimator on the joint statistic T(X) but in effect yes, you got it. $\endgroup$ – Nick Thieme Jul 30 '15 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.