# Why is median not a sufficient statistic? [duplicate]

Suppose a random sample of $$n$$ variables from $$N(\mu,1)$$, $$n$$ odd. The sample median is $$M=X_{(n+1)/2}$$, the order statistic of the middle of the distribution.

How to prove that sample median is not a sufficient statistic of $$\mu$$? Do we need the probability density function of M?

I would like any help.

Note 1: I'm studying for the final exam of my course, and take previous exams to practice. This question appeared in a 2015 exam, that was given by other professor than mine, so it is possible that covers things that wasn't given by my professor. This question includes others two exercises: (1) Is $$M-\bar{X}$$ an ancillary statistic for $$\mu$$? (2) Prove that if $$m$$ is a unique point, such as $$F_X(m)=1/2$$, then $$M \overset{p}{\rightarrow}m$$. I couldn't answer these two exercises, because ancillary statistics were not covered by my professor.

So, to see wheter $$M$$ is a sufficient statistic for $$\mu$$, I derived the probability density (pdf) function of $$M$$, that I bring below. But I think there is a more inteligent way to do, without getting the pdf first. So this is the reason I'm asking this question, to see if there is a better and faster way of solving this question.

Note 2: things I know about sufficient statistics:

$$T$$ is a sufficient statistic if $$\frac{f_X(x|\theta)}{q(t|\theta)}$$ does not depend on $$\theta$$, where $$f_X$$ is the probability density function of $$X$$ and $$q$$ is the probability density function of $$T$$. Also, $$T$$ is a sufficient statistic if, and only if, there exist functions $$g(t|\theta)$$ and $$h(x)$$ such that $$f_X(x|\theta)=g(T(x)|\theta)h(x)$$ for all points of $$\theta$$

Note 3: The probability density function of M is

$$f_{M}(m)=\frac{n!}{\left ( \frac{n-1}{2} \right) ! \left ( \frac{n-1}{2} \right) !}[1-F_{X}(m)]^{\frac{n-1}{2}}f_{X}(m)[F_X(m)]^{\frac{n-1}{2}}$$

I divided the probability density functions of $$(X_1,...,X_n)$$ and $$M$$: $$\frac{f_{X}(x|\mu)}{f_{M}(m)}=\frac{f_{X1}(x_{1}|\mu)\cdot ...\cdot f_{Xn}(x_{n}|\mu) }{\frac{n!}{\left ( \frac{n-1}{2} \right) ! \left ( \frac{n-1}{2} \right) !}[1-F_{X}(m)]^{\frac{n-1}{2}}f_{X}(m)[F_X(m)]^{\frac{n-1}{2}}}$$

So we have (I don't know if is wright): $$\frac{f_{X}(x|\mu)}{f_{M}(m)}=\frac{f_{X1}(x_{1}|\mu)\cdot ...\cdot f_{X_{n-1}}(x_{n-1}|\mu) }{\frac{n!}{\left ( \frac{n-1}{2} \right) ! \left ( \frac{n-1}{2} \right) !}[1-F_{X}(m)]^{\frac{n-1}{2}}[F_X(m)]^{\frac{n-1}{2}}}$$ It seems that this expression depends of $$\mu.$$

• 1. This appears to be a homework-like question; please see stats.stackexchange.com/tags/self-study/info . 2. What things do you know about sufficient statistics? e.g. definition/facts/results/theorems etc? 3. What have you tried in order to use the things you know to answer the question? Commented Feb 22, 2023 at 1:53
• $T$ is a sufficient statistic if $$\frac{f_X(x|\theta)}{q(t|\theta)}$$ does not depend on $\theta$, where $f_X$ is the probability density function of $X$ and $q$ is the probability density function of $T$ Also, $T$ is a sufficient statistic if, and only if, there exist functions $g(t|\theta)$ and $h(x)$ such that $$f_X(x|\theta)=g(T(x)|\theta)h(x)$$ for all points of $\theta$ Commented Feb 22, 2023 at 2:00
• That's definitely a good thing to know. Did you try to use it? However, is that the only thing you have? 4. You should consider how $M$ is related to the parameter of the distribution of the $X_i$, $\mu$. i.e. note that $F$ and $f$ involve $\mu$ Commented Feb 22, 2023 at 2:01
• I'm not saying you need to use that fact, but if you just write it in terms of F and f without thinking about how those things relate to $\mu$ you may not even see that $\mu$ is present there and trick yourself into thinking it isn't. Commented Feb 22, 2023 at 2:07
• I divided the probability density functions of $(X_1,...,X_n)$ and $M$: $$\frac{f_{X}(x|\mu)}{f_{M}(m)}=\frac{f_{X1}(x_{1}|\mu)\cdot ...\cdot f_{Xn}(x_{n}|\mu) }{\frac{n!}{\left ( \frac{n-1}{2} \right) ! \left ( \frac{n-1}{2} \right) !}[1-F_{X}(m)]^{\frac{n-1}{2}}f_{X}(m)[F_X(m)]^{\frac{n-1}{2}}}$$ So we have (I don't know if is wright): $$\frac{f_{X}(x|\mu)}{f_{M}(m)}=\frac{f_{X1}(x_{1}|\mu)\cdot ...\cdot f_{X_{n-1}}(x_{n-1}|\mu) }{\frac{n!}{\left ( \frac{n-1}{2} \right) ! \left ( \frac{n-1}{2} \right) !}[1-F_{X}(m)]^{\frac{n-1}{2}}[F_X(m)]^{\frac{n-1}{2}}}$$ that depends of $\mu$ Commented Feb 22, 2023 at 2:16

A tricky part of this question is that the median sounds like a good estimator and mathematically it is not so clear what the mean is gonna improve.

So imagine the following simpler case:

• Say we have a sample of size $$X_1, X_2, \dots , X_n$$ where each $$X_i$$ follows (independently) a normal distribution $$X_i \sim N(\theta,1)$$ and we want to estimate $$\theta$$.
• We could make an estimate $$\hat{\theta} = X_1$$ and it is obvious that this is not a sufficient statistic. The other $$n-1$$ values can provide information about $$\theta$$ as well. The statistic $$X_1$$ is not sufficient.
• A sufficient statistic occurs when the distribution of the data is independent of the parameter $$\theta$$ conditional on the sufficient statistic. This is not true for $$X_1$$. For example, the distribution of $$\bar{X}$$ conditional on $$X_1$$ is not independent from $$\theta$$

One obvious way to check if a statistic is sufficient is to identify a minimal sufficient statistic (if it exists) and check if the minimal sufficient statistic is a function of your proposed statistic. Here we want to use the fact that a minimal sufficient statistic is a function of any sufficient statistic.

Take $$n=3$$ for example. A minimal sufficient statistic for $$\mu$$ is the sample mean $$\overline x=\frac13(x_1+x_2+x_3)$$. But $$\overline x$$ is not a function of the sample median $$x_{(2)}$$ only. You have to argue this formally.

In essence, sufficiency is concerned with data reduction (see What does it mean that a statistic $T(X)$ is sufficient for a parameter?). So $$\overline x$$ is sufficient means that all the information about $$\mu$$ in the sample can be condensed in $$\overline x$$. This happens to be the maximum possible data condensation in this model, which makes the sample mean minimal sufficient. Given $$\overline x$$, you can make inference on $$\mu$$. But knowing $$x_{(2)}$$ alone does not give you enough information about $$\mu$$. You also need to know $$x_{(1)}$$ and $$x_{(3)}$$ to avoid any loss of information. A numerical example might help to make this point clear.

This is a great technical question. First I want to point out that your argument

$$T$$ is a sufficient statistic if $$\frac{f_X(x|\theta)}{q(t|\theta)}$$ does not depend on $$\theta$$, where $$f_X$$ is the probability density function of $$X$$ and $$q$$ is the probability density function of $$T$$.

does not hold. To see it, we know that for $$T = \bar{X} \sim N(\mu, \frac{1}{n})$$, whence \begin{align} & \frac{f_\mu(x_1, \ldots, x_n)}{q_\mu(t)} = \frac{(2\pi)^{-n/2}\exp(-\frac{1}{2}\sum_{i = 1}^n(x_i - \mu)^2)} {(2\pi n)^{-1/2}\exp(-\frac{n}{2}(t - \mu)^2)} = h(x)e^{-\frac{n}{2}(\bar{x} - t)\mu}, \end{align} which depends on $$\mu$$ (you may argue that it does not depend on $$\mu$$ if substitute $$\bar{x}$$ to $$t$$, however, this is not what the proposed ratio formula says), but $$\bar{X}$$ is of course sufficient. It looks like your "criterion" tries to match the definition of sufficiency of $$T$$ (Section 1.6, Theory of Point Estimation):

A statistic $$T$$ is said to be sufficient for $$X$$, or for the family $$\mathcal{P} = \{P_\theta, \theta \in \Omega\}$$ of possible distributions of $$X$$, or for $$\theta$$, if the conditional distribution $$X$$ given $$T = t$$ is independent of $$\theta$$ for all $$t$$.

A naive interpretation of the above definition is that the conditional density of the data $$X = (X_1, \ldots, X_n)$$ given $$T = t$$ is independent of $$\theta$$, which may be formally written as (note how the numerator in $$(1)$$ differs from that in your proposed ratio) \begin{align} f_{X|T; \theta}(x|t) = \frac{f_\theta(x, t)}{q_\theta(t)} \text{ is independent of \theta.} \tag{1} \end{align} However, a technical difficulty of the interpretation $$(1)$$ is that the "joint density" $$f_\theta(x, t)$$ of $$(X, T)$$ is degenerate (i.e., it is not a probability density on $$\mathbb{R}^{n + 1}$$. In general, the conditional density defining relation $$f_{X|Y = y}(x|y) = \frac{f(x, y)}{f_Y(y)}$$ only makes sense when $$f(x, y)$$ is a valid, non-degenerate density), hence $$(1)$$ is actually impossible to check. For a relevant discussion, see this question. To solve this difficulty, some 1-1 transformation between $$(X_1, \ldots, X_n)$$ and $$(Y_1, \ldots, Y_{n - 1}, T)$$ needs to be defined and the sufficiency needs to be augmented accordingly in terms of the conditional density of $$Y$$ given $$T$$. For details, refer to Eq (1.17) -- Eq (1.19) in Section 1.9, Testing Statistical Hypotheses.

Having clarified this, one way to show that $$M$$ is not a sufficient statistic for $$\mu$$ is to find a subset $$Y$$ (possibly with transformation) of $$(X_1, \ldots, X_n)$$, such that the conditional density of $$Y$$ given $$T$$ is well-defined (i.e., non-degenerate) and does depend on $$\theta$$. For simplicity and without losing of generality, assume $$n = 3$$.

One choice of $$Y$$ is $$(X_{(1)}, X_{(3)}) = (\min(X_1, X_2, X_3), \max(X_1, X_2, X_3))$$. It is well known by the order statistic theory that the joint density of $$(Y, M) = (X_{(1)}, X_{(3)}, X_{(2)})$$ is \begin{align} f(x_{(1)}, x_{(3)}, x_{(2)}) = 6\varphi_\mu(x_{(1)})\varphi_\mu(x_{(2)})\varphi_\mu(x_{(3)}), \quad x_{(1)} < x_{(2)} < x_{(3)}, \tag{2} \end{align} where $$\varphi_\mu(x) = \frac{1}{\sqrt{2\pi}}e^{-\frac{(x - \mu)^2}{2}}$$. And the marginal density of $$X_{(2)}$$ is (where $$\Phi_\mu(x) = \int_{-\infty}^x \varphi_\mu(t)dt$$) \begin{align} f_{X_{(2)}}(x_{(2)}) = 6\Phi_\mu(x_{(2)})(1 - \Phi_\mu(x_{(2)}))\varphi_\mu(x_{(2)}). \tag{3} \end{align} $$(2)$$ and $$(3)$$ then give the conditional density of $$(X_{(1)}, X_{(3)})$$ given $$X_{(2)}$$ as \begin{align} f_{(X_{(1)}, X_{(3)})|X_{(2)} = x_{(2)}}(x_{(1)}, x_{(3)}|x_{(2)}) = \frac{\varphi_\mu(x_{(1)})\varphi_\mu(x_{(3)})} {\Phi_\mu(x_{(2)})(1 - \Phi_\mu(x_{(2)}))}, \end{align} which depends on $$\mu$$.

In contrast, you can verify that \begin{align} f_{(X_{(1)}, X_{(3)})|\bar{X} = \bar{x}}(x_{(1)}, x_{(3)}|\bar{x}) = \frac{3\sqrt{3}}{\pi}\exp\left(\frac{3}{2}\bar{x}^2 - \frac{1}{2}(x_{(1)}^2 + x_{(3)}^2 + (3\bar{x} - x_{(1)} - x_{(3)})^2)\right), \end{align} which is independent of $$\mu$$.

Let us assume that $$n = 2k+1$$ is odd, and let $$J_i$$ be a ternary variable showing whether $$X_i$$ is equal to the median $$M$$, below it or above it. That is, $$J_i = \begin{cases} 1 & \text{X_i > M} \\ 0 & \text{X_i = M} \\ -1 & \text{X_i < M} \end{cases}$$ If $$M$$ is sufficient, then $$(M,J_1,\dots,J_n)$$ has to be sufficient as well. So it is enough to show that $$(M,J_1,\dots,J_n)$$ is not sufficient. What does sufficiency mean? That the distribution of $$(X_1,\dots,X_n)$$ given $$(M,J_1,\dots,J_n))$$ is independent of the parameter, $$\mu$$ in this case. That this independence fails in this case is intuitive, but a bit cumbersome to write down.

With probability 1, exactly one of $$J_1,\dots,J_n$$ can be equal to 0, and the rest divided equally among +1 and -1 (since $$N(\mu,1)$$ is continuous w.r.t. the Lebesgue measure the chance of seeing repeated values in a finite sequence of independent draws from it is zero.) Without loss of generality, let us condition on \begin{align} A = \{M = m,\;\; J_1 = 0, \;\;&J_2 = +1, \dots, J_{k+1} = +1, \\ &J_{k+2} = -1, \dots, J_{2k+1} = -1\}, \end{align} that is, $$X_1$$ is the median, the next $$k$$ observations are below the median and the next $$k$$ after are above the median.

Then, $$X_1 = m$$ is completely determined, and by independence $$X_2,\dots,X_{k+1}$$ are i.i.d. from $$N(\mu,1)$$ truncated to $$(m,\infty)$$ which we denote as $$N(\mu,1; m, \infty)$$. Similarly, $$X_{k+2},\dots,X_{2k+1}$$ are i.i.d. from $$N(\mu,1)$$ truncated to $$(-\infty,m)$$ which we denote as $$N(\mu,1; -\infty, m)$$. Clearly, the distribution of $$(X_1,\dots,X_n)$$ given $$A$$ depends on $$\mu$$ and hence sufficiency fails.

If you want to write it down, technically, $$(X_1,\dots,X_n) \, |\, A \;\;\sim \;\; \delta_{m} \otimes \Bigl(\prod_{i=1}^k N(\mu,1; m, \infty) \Bigr) \otimes \Bigl(\prod_{i=1}^k N(\mu,1; -\infty, m) \Bigr)$$ where $$\delta_m$$ is the point mass measure at $$m$$, and $$\otimes$$ and $$\prod$$ denote products of measures. You can also consider other combinations of values for $$\{J_i\}$$ and it would be similar (you just get a different permutation of the same terms), but not necessary to establish the failure of sufficiency (one combination is enough).

You can also answer the question about ancillarity. A statistic is ancillary if its distribution does not depend on the parameter $$\mu$$.