13
$\begingroup$

Consider a sample $X_1,X_2,\ldots,X_n$ from a univariate $N(\mu,\sigma^2)$ distribution where $\mu,\sigma^2$ are both unknown. Then it is known that under squared error loss, the sample variance $s^2=\frac1{n-1}\sum\limits_{i=1}^n (X_i-\overline X)^2$ is inadmissible for estimating $\sigma^2$ because there is a better estimator $\left(\frac{n-1}{n+1}\right)s^2=\frac1{n+1}\sum\limits_{i=1}^n(X_i-\overline X)^2$.

Now is this second estimator itself admissible under the same loss function? It certainly has the minimum risk among estimators of the form $cs^2$, but how do we know there isn't another estimator outside this class with a smaller risk?

I have the same question for when $\mu$ is known. If $\mu=0$, then it can be shown that $T=\frac1n\sum\limits_{i=1}^n X_i^2$ is not admissible under squared error loss for estimating $\sigma^2$ because there is a better estimator $\left(\frac{n}{n+2}\right)T=\frac1{n+2}\sum\limits_{i=1}^n X_i^2$. But I don't know if $\left(\frac{n}{n+2}\right)T$ is admissible or not.


I finally found an accessible reference for both these problems in Lehmann/Casella's Theory of Point Estimation (2nd ed, pages 330-334).

$\endgroup$
0

1 Answer 1

8
$\begingroup$

Assume that $X_i \sim \mathcal{N}(\mu, \sigma^2)$ are i.i.d. with unknown $\mu$ and $\sigma^2$ and that the loss function is $\left(\delta(\mathbf{X}) - \sigma^2 \right)^2$. Consider the reference estimator $\delta_0(\mathbf{X}) = \sum_{i=1}^n (X_i-\bar{X})^2/(n+1)$.

Stein (1964) found an estimator $\delta_{(\nu)}$ which dominates $\delta_0$ for any fixed choice $\nu$. The estimator is $$\delta_{(\nu)}(\mathbf{X}) = \min\left\{ \delta_0(\mathbf{X}), \frac{1}{n+2} \sum_{i=1}^n (X_i - \nu)^2\right\}.$$ At a high level, this estimator chooses between an estimator with unknown mean (estimated with the sample mean) and an estimator which commits to the fixed $\nu$ as being the true mean and thus has one more df. Stein writes that ``It is interesting to observe that the estimator [$\delta_{(\nu)}$] may be obtained by first testing the hypothesis $\mu=\nu$ at an appropriate significance level and using the estimate [$\frac{1}{n+2} \sum_{i=1}^n (X_i - \nu)^2$] if the hypothesis is accepted and the estimate [$\delta_0(\mathbf{X})$] if the hypothesis is rejected.''

From my perspective, the estimator $\delta_{(\nu)}$ bets on $\nu$ being the true mean and is allowed to ``shirk'' out of the bet. This is a kind of post selection inference which is common in modern high dimensional statistics, but here there's no need to control for the selection.

Note, I was initially surprised to see this from Stein since in my experience, estimators for covariances are (sensibly) compared with scale invariant loss functions. On this point, in the linked article, Stein writes "I find it hard to take the problem of estimating a [variance] with quadratic loss function very seriously" and goes on to elaborate and conclude that "unlike the results of the present paper, the main results in [another paper which uses a squared error loss but for location estimation] can be seriously recommended to the practical statistician".


In the case that the true mean $\mu$ is known, the estimator with division by $n+2$ is admissible under squared error. This is reviewed in the above article from Stein and credited to Hodges and Lehmann (1951).

$\endgroup$
9
  • 1
    $\begingroup$ I found a free version of the article: apps.dtic.mil/dtic/tr/fulltext/u2/1028390.pdf. $\endgroup$
    – Dave
    May 10, 2021 at 16:19
  • $\begingroup$ Super interesting answer! My immediate thought was that the scale invariant estimator would be admissible in this problem, by analogy with what happens with the location-invariant estimator of the mean in low dimensions, while in higher dimensions it would not be admissible. $\endgroup$
    – guy
    May 10, 2021 at 16:35
  • $\begingroup$ Thank you. I am tempted to ask if any estimator of the form $c\sum_i (X_i-\overline X)^2$ is admissible. And is there any discussion in this paper of the same problem for the multivariate normal? $\endgroup$ May 10, 2021 at 16:56
  • $\begingroup$ @Dave That looks like a different article. $\endgroup$ May 10, 2021 at 16:56
  • 1
    $\begingroup$ @StubbornAtom You've already found the best estimator of that form, and this shows its inadmissible. Therefore all such estimators are inadmissible. $\endgroup$
    – user257566
    May 10, 2021 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.