6
$\begingroup$

I always read that every maximum likelihood estimator has to be a function of any sufficient statistic. The idea is that, if we are dealing with a random variable $X$ with mass or density function $f(x\mid\theta)$, and $T$ is a sufficient statistic for $\theta$, then by the factorization theorem $f(\vec{x}\mid\theta)=g(T(\vec{x}),\theta)h(\vec{x})$, so maximizing $f(\vec{x}\mid\theta)$ on $\theta$ means maximizing $g(T(\vec{x})\mid\theta)$ on $\theta$, therefore every maximum likelihood estimator for $\theta$ must be a function of $T(\vec{x})$.

However, I have the following counterexample? for this result:

Let $X\sim\text{Unif}(\theta-1/2,\theta+1/2)$. The likelihood function if $L(\theta\mid\vec{x})=1_{[x_{(n)}-1/2,x_{(1)}+1/2]}$, where $x_{(1)}$ and $x_{(n)}$ are, respectively, the minimum and the maximum of our sample $\vec{x}$ of size $n$. Then, any $\hat{\theta}$ with $x_{(n)}-1/2\leq\hat{\theta}\leq x_{(1)}+1/2$ is a maximum likelihood estimator. Also, note that $(X_{(1)},X_{(n)})$ is a sufficient statistic. Now let $$\hat{\theta}=x_{(n)}-1/2+\frac{|x_j|}{1+|x_j|}(x_{(1)}-x_{(n)}+1),$$ where $x_j\neq x_{(1)}$ and $x_j\neq x_{(n)}$. This $\hat{\theta}$ is a maximum likelihood estimator for $\theta$, but is not a function of $(x_{(1)},x_{(n)})$.

What is wrong?

Improve this question
$\endgroup$
17
  • 2
    $\begingroup$ I'm sorry, but why is this not a function of $(x_{(1)},x_{(n)})$? Your mathematical expression clearly says it is a function! $\endgroup$
    – Landon Carter
    Commented Dec 28, 2016 at 18:07
  • 1
    $\begingroup$ It seems that your example deals with the uniqueness of the MLE and not its dependence on the sufficient statistic. $\endgroup$
    – Michael R. Chernick
    Commented Dec 28, 2016 at 18:14
  • 1
    $\begingroup$ In English "the" means 1 "a" means possibly more than one. This is technically not in the "proof" but is in the problem statement (proposition). $\endgroup$
    – Michael R. Chernick
    Commented Dec 28, 2016 at 20:00
  • 6
    $\begingroup$ I would not be inclined to answer this question unless some authoritative source of the incorrect assertions on which it relies can be cited. In particular, the introductory claim "every maximum likelihood estimator has to be a function of any sufficient statistic" is not true. What is true is that every ML estimator, if unique, must be a function of the sufficient statistics. This is an immediate consequence of the definitions of the MLE and sufficient statistic. (See Kendall & Stuart 5th Ed. Vol II section 18.4.) $\endgroup$
    – whuber ♦
    Commented Dec 29, 2016 at 15:50
  • 4
    $\begingroup$ You erred at the point you wrote "therefore". Your conclusion does not follow unless the maximum value depends uniquely on $T$, for otherwise you are free to select $\hat\theta$ in arbitrary ways--exactly as you do in your example. $\endgroup$
    – whuber ♦
    Commented Dec 29, 2016 at 19:02

3 Answers 3

Reset to default
8
$\begingroup$

Nothing is wrong with what you said, just the statement that every maximum likelihood estimator has to be a function of any sufficient statistic, which is false as stated. A more correct form of putting this assertion is:

If $T$ is a sufficient statistic for $\theta$ and a unique MLE of $\hat{\theta}$ exists, then $\hat{\theta}$ must be a function of $T$. If any MLE exists, then an MLE $\hat{\theta}$ can be chosen to be a function of $T$.

This quote is from Maximum Likelihood and Sufficient Statistics found in The American Mathematical Monthly by D.S. Moore. You can find it on JSTOR. You can also find an example similar to yours and more information about your question.

Improve this answer
$\endgroup$
2
$\begingroup$

I think to preserve the theorem in cases like this one should define the MLE as the interval of MLEs. That is a function of the sufficient statistic.

This page takes a different point of view: For every sufficient statistic, there is at least one MLE that is a function of it. (So if there is only one MLE, then that one is it.)

Improve this answer
$\endgroup$
1
$\begingroup$

Sufficient statistics only apply to exponential family distributions. Continuous uniform is not an exponential family distribution. See [DeGroot, Morris H. Optimal Statistical Decisions, 1970, McGraw-Hill Book Company, New York]

Improve this answer
$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.