33
$\begingroup$

I came across a casual remark on The Chemical Statistician that a sample median could often be a choice for a sufficient statistic but, besides the obvious case of one or two observations where it equals the sample mean, I cannot think of another non-trivial and iid case where the sample median is sufficient.

$\endgroup$
5
  • 1
    $\begingroup$ Did you mean to write "that a sample median could often be"? $\endgroup$ Nov 6, 2014 at 13:55
  • 10
    $\begingroup$ It's an interesting question; the double exponential has the median for a ML estimator of its location parameter, but it's not sufficient. $\endgroup$
    – Glen_b
    Nov 6, 2014 at 15:57
  • 1
    $\begingroup$ honestly, i strongly feel that something is missing in this Q&A, how is it possible that a ML estimator is not sufficient for itself? sorry for just throwing my doubts this way, i was never really interested in sufficient statistics. $\endgroup$
    – carlo
    Aug 15, 2020 at 7:28
  • $\begingroup$ @carlo: what do you mean by "sufficient for itself"? $\endgroup$
    – Xi'an
    Aug 15, 2020 at 19:47
  • $\begingroup$ @Xi’an, I deleted my comment, but I’ll see if I can turn the ideas into a proper answer. $\endgroup$
    – Matt F.
    Jun 29, 2021 at 10:33

2 Answers 2

25
+50
$\begingroup$

In the case when the support of the distribution does not depend on the unknown parameter $\theta, $ we can invoke the (Fréchet-Darmois-)Pitman-Koopman theorem, namely that the density of the observations is necessarily of the exponential family form, $$ \exp\{ \theta T(x) - \psi(\theta) \}h(x) $$ to conclude that, since the natural sufficient statistic $$ S=\sum_{i=1}^n T(x_i) $$ is also minimal sufficient, then the median should be a function of $S$, and the other way as well, which is impossible: modifying an extreme in the observations $x_1,\ldots,x_n$, $n>2$, modifies $S$ but does not modify the median. Therefore, the median cannot be sufficient when $n>2$.

In the alternative case when the support of the distribution does depend on the unknown parameter $θ$, I am less happy with the following proof: first, we can wlog consider the simple case when $$ f(x|\theta) = h(x) \mathbb{I}_{A_\theta}(x) \tau(\theta) $$ where the set $A_\theta$ indexed by $θ$ denotes the support of $f(\cdot|\theta)$. In that case, assuming the median is sufficient, the factorisation theorem implies that we have that $$ \prod_{i=1}^n \mathbb{I}_{A_\theta}(x_i) $$ is a binary ($0-1$) function of the sample median $$ \prod_{i=1}^n \mathbb{I}_{A_\theta}(x_i) = \mathbb{I}_{B^n_\theta}(\text{med}(x_{1:n})) $$ Indeed, there is no extra term in the factorisation since it should also be (i) a binary function of the data and (ii) independent from $\theta$. Adding a further observation $x_{n+1}$ which value is such that it does not modify the sample median then leads to a contradiction since it may be in or outside the support set, while $$ \mathbb{I}_{B^{n+1}_\theta}(\text{med}(x_{1:n+1}))=\mathbb{I}_{B^n_\theta}(\text{med}(x_{1:n}))\times \mathbb{I}_{A_\theta}(x_{n+1}). $$

$\endgroup$
14
  • 4
    $\begingroup$ Isn't median sufficient for Laplace when scale is known? $\endgroup$ Jan 6, 2020 at 13:15
  • 2
    $\begingroup$ No, it is not. Because Laplace is not within exponential families. $\endgroup$
    – Xi'an
    Jan 6, 2020 at 13:38
  • 2
    $\begingroup$ You say and the case is solved. I am just wondering what the conclusion is. Is it that in this setting (support not depending on $\theta$), median is a sufficient statistic? Or is not a sufficient statistic? What about the second case? What do we learn about median as a sufficient statistic from the two cases taken together? I am not questioning the technical points you make, but I am trying to extract a takeaway message, an answer to the question you raise in the OP. You may think everything is obvious from what you have already written, but it might not be obvious for everyone (e.g. me). $\endgroup$ Aug 13, 2020 at 7:05
  • 3
    $\begingroup$ The answer to the question is that the median cannot be a sufficient statistic, except in trivial cases (like $n=1$). $\endgroup$
    – Xi'an
    Aug 13, 2020 at 7:08
  • 2
    $\begingroup$ @SextusEmpiricus To have the Uniform density written as an exponential family density $$f(x) \propto h(x) \cdot \exp(\eta(\theta) \cdot T(x))$$ is not feasible, meaning that most analytic properties associated with exponential families are not extendable to the Uniform distribution. $\endgroup$
    – Xi'an
    Feb 24, 2023 at 7:16
4
$\begingroup$

Xi'an's answer raises the question of what is happening with the Laplace (double exponential distribution), where the MLE is the median. As Xi'an says, the median is not sufficient in this model; it can't be, because among families with constant support only exponential families have finite-dimensional sufficient statistics.

The log likelihood looks like this enter image description here

It's piecewise linear, increasing below the median and decreasing above the median. Note that it has a 'corner' at each observed $x$; you can see it doesn't factorise into a term that doesn't depend on parameter $\theta$ and a term that depends only on $\theta$ and the median.

Now, why is this possible? The MLE is necessarily a function of any sufficient statistic, but it's not necessarily true that there's a sufficient statistic that's a function of the MLE. This seems to imply there's information in the data that doesn't make it into the MLE. Whether that's true or not depends on how you define 'information in the data', but this paper shows that the median fails to attain the Cramer-Rao bound for the Laplacian model by a factor of about $1+8/n$.

What is true about the MLE is that it's asymptotically efficient. It's also asymptotically sufficient in some sense. For example, the Convolution theorem says that (under weak conditions) any estimator is equal to the asymptotically efficient estimator plus noise that doesn't depend on the parameter. Lin and Zeng's paper showing there's no asymptotic efficiency loss in meta-analysis using summary statistics is also relevant here

$\endgroup$
2
  • $\begingroup$ (+1) The M.L.E. isn't always asymptotically efficient, though there are some regularity conditions that will guarantee it is. Estimation of $\theta$ for i.i.d. r.v.s distributed uniformly between 0 & $\theta$ provides a good counter-example. $\endgroup$ Feb 24, 2023 at 19:04
  • $\begingroup$ @Scortchi-ReinstateMonica: in that Uniform case, it is super-efficient. $\endgroup$
    – Xi'an
    Feb 25, 2023 at 11:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.