8
$\begingroup$

My homework problem is to give a counterexample where a certain statistic is not in general minimal sufficient. Irrespective of the details of finding a particular counterexample for this particular statistic, this raises the following question for me:

Question: How can one formulate the condition of not being a minimal sufficient statistic in a way that is possible to prove that a sufficient statistic satisfies the condition?

Work so far: The definition of minimal sufficient statistic in my textbook (Keener, Theoretical Statistics: Topics for a Core Course) is as follows:

  • A statistic $T$ is minimal sufficient iff $T$ is sufficient and, for every sufficient statistic $\tilde{T}$ there exists a function $f$ such that $T = f(\tilde{T})$ a.e. $\mathcal{P}$.

Note that (a.e. $\mathcal{P}$) means that the set where equality fails is a null set for every probability distribution $P$ in the statistical model $\mathcal{P}$, $P \in \mathcal{P}$.

Trying to negate this, I arrive at:

  • A statistic $T$ is not minimal sufficient iff at least one of the following holds:
    1. $T$ is not sufficient.
    2. There exists at least one sufficient statistic $\tilde{T}$ for which there is no function $f$ such that $T = f(\tilde{T})$ a.e. $\mathcal{P}$.

So if a statistic is sufficient, then it seems like it would be extremely difficult to show that it is not minimal sufficient, even if it is not minimal sufficient. (Because one would have to show 2. instead of 1., since 1. is false -- but 2. would be very difficult to show because, even if one has a counterexample statistic $\tilde{T}$ in mind, one still has to show the non-existence of any function with that property. And non-existence is often difficult to show.)

My textbook does not give any equivalent (i.e. necessary and sufficient) conditions for a statistic to be a minimal sufficient statistic. It does not even give any alternative necessary conditions for a statistic to be minimal sufficient statistic (besides being a sufficient statistic).

Therefore, for my homework problem, if I can't show that the statistic is not sufficient (because it is), then how could I ever possibly show that it is not minimal sufficient?

$\endgroup$
9
  • 3
    $\begingroup$ Have you considered starting with a minimal sufficient statistic and then enlarging it to include more components? $\endgroup$
    – whuber
    Commented Sep 6, 2017 at 18:20
  • 4
    $\begingroup$ In mathematics in general, one often proves the nonexistence of something by assuming it exists and using it to find a contraction. $\endgroup$ Commented Sep 6, 2017 at 18:24
  • 2
    $\begingroup$ A statistic is a vector-valued function of the data. It has components. For instance, a minimal sufficient statistic for the Normal family of distributions is the two-vector consisting of the sample mean and sample variance. Adjoining more components--throw in the sample skewness and kurtosis, for instance--gives you a statistic with four components. My hint merely stated the obvious: this new statistic obviously is sufficient, because its first two components already are sufficient. But is it minimal sufficient? $\endgroup$
    – whuber
    Commented Sep 6, 2017 at 19:10
  • 2
    $\begingroup$ I don't see how any of those observations about bijections or homeomorphisms could possibly be relevant. Are you using some unusual definition of "statistic" or "sufficient"? $\endgroup$
    – whuber
    Commented Sep 6, 2017 at 19:21
  • 3
    $\begingroup$ You seem to be using some kind of unconventional definition of sufficiency. In my example, all that matters is that the new statistics are genuine statistics--measurable functions of the data. The map from $\mathbb{R}^4$ to $\mathbb{R}^2$ (that retrieves the original two statistics, the minimal sufficient one) is measurable (indeed, differentiable). That's all you have to check. $\endgroup$
    – whuber
    Commented Sep 6, 2017 at 22:36

2 Answers 2

3
$\begingroup$

As you stated:

If there exist $x_1,x_2\in X$ such that $f(x_1)=f(x_2)$ but $g(x_1)\ne g(x_2)$, then $g$ can not be written as a function of $f$, i.e. there exists no function $h$ with $g=h∘f$.

So, for example, in the case where $X_1, ...., X_n$ are independent Bernoulli random variables. We can prove that $(x_1, ...., x_n)$ is not minimally sufficient by showing that it is not a function of $\sum x_i$. This is obvious, since the function must map $1$ to both $(1,0,0...,0,0,0)$ and $(0,0,0...,0,0,1)$.

$\endgroup$
2
$\begingroup$

I have been thinking about this problem some more recently, and here is what I have come up with.

Let $\Omega$ be a probability space, then a random variable $X$ is a measurable function $X: \Omega \to \mathcal{X}$, where $\mathcal{X}$ is a measurable space ($\mathcal{X}$ has a designated $\sigma$-algebra, and $X$ is measurable with respect to this $\sigma$-algebra and the $\sigma$-algebra on $\Omega$). The distribution of $X$ is just the pullback measure on $\mathcal{X}$, i.e. $\mathbb{P}_{\mathcal{X}}(A) = \mathbb{P}_{\Omega}(X^{-1}(A))$. Then a statistic of $X$ is any measurable* function $f: \mathcal{X} \to \mathcal{Y}$, where $\mathcal{Y}$ is another arbitrary measurable space.

Given two statistics $f: \mathcal{X} \to \mathcal{Y}$, $g: \mathcal{X} \to \mathcal{Z}$, what does it mean for "$g$ to be a function of $f$"?

As far as I can tell, it seems to mean that there exists a measurable** function $h: \mathcal{Y} \to \mathcal{Z}$ such that $g = h \circ f$, i.e. that $g$ can be factored through by $f$.

(In other words, "$g$ must be well-defined as a function on $f(\mathcal{X}) \subseteq \mathcal{Y}$".)

So when is such factoring possible? Let's think in terms of equivalence relations. Specifically, define the equivalence relation $\sim_f$ on $\mathcal{X}$ by $x_1 \sim_f x_2 \iff f(x_1) = f(x_2)$, likewise, define the equivalence relation $\sim_g$ on $\mathcal{X}$ by $x_1 \sim_g x_2 \iff g(x_1) = g(x_2)$.

Then in order for $g$ to be factorable by $f$, the equivalence relations $\sim_f$ and $\sim_g$ need to be compatible with each other, in the sense*** that for any $x_1, x_2 \in \mathcal{X}$, $x_1 \sim_f x_2 \implies x_1 \sim_g x_2$, i.e. $g$ can't take two elements which are equivalent under $f$ and map them to values which aren't equivalent under $g$, i.e. "$g$ can't undo the information reduction previously performed by $f$".

In other words, $g$ has to be well-defined as a function on $\mathcal{X}/\sim_f \cong f(\mathcal{X})$, i.e. there exists has to exist a function $\tilde{g}: \mathcal{X}/\sim_f \to \mathcal{Z}$ such that $g = \tilde{g} \circ \pi_f$, where $\pi_f$ is the canonical projection $\mathcal{X} \to \mathcal{X}/\sim_f$. (For those uncomfortable with abstract non-sense, $\pi_f$ is essentially $f$, and $\tilde{g}$ is essentially $h$. The above formulation just makes analogies with other situations more clear.)

In simplest possible words, $g$ can be written as function of $f$ if and only if, for any $x_1, x_2 \in \mathcal{X}$, $f(x_1) = f(x_2) \implies g(x_1) = g(x_2)$.

For example, take $\mathcal{X} = \mathcal{Y} = \mathcal{Z} = \mathbb{R}$ and $X$ an arbitrary real-valued random variable, then $g: x \mapsto x^2$ can be written as a function of $f: x \mapsto x$, but not vice versa, because $x_1 = x_2 \implies x_1^2 = x_2^2$, but $1^2 = (-1)^2$ but $1 \not= -1$.

In particular, assume that every equivalence class under $\sim_f$ is a singleton (i.e. $f$ is injective). Then $g$ can always be written as a function of $f$, since $\mathcal{X}/\sim_f \cong \mathcal{X}$, i.e. $f(x_1) = f(x_2) \implies x_1 = x_2$ means that $x_1 = x_2 \iff f(x_1) = f(x_2)$ (in general, for not-necessarily injective $f$, only one direction holds), so our condition becomes $x_1 = x_2 \implies g(x_1) = g(x_2)$, which is trivially satisfied for any $g: \mathcal{X} \to \mathcal{Z}$. (To define $h$, it can do anything it wants on $\mathcal{Y} \setminus f(\mathcal{X})$ as long as it's measurable, and then for any $y \in f(\mathcal{X})$, i.e. such that $y = f(x)$ for some $x \in \mathcal{X}$, define $h$ to be $h: y = f(x) \mapsto g(x)$. This is well-defined when $f$ is injective because there is a unique $x \in \mathcal{X}$ such that $f(x) = y$. More generally, this is only defined when, regardless of which $x$ we choose in $f^{-1}(y)$, $g(x)$ still is the same value, i.e. $f(x_1)=f(x_2)\ (=y) \implies g(x_1)=g(x_2)$.)

Also, looking at Theorem 3.11 in Keener, its statement is kind of clunky, but thinking in the above terms, I believe it can be re-written as:

Suppose $T$ is a sufficient statistic****. Then a sufficient condition for $T$ to be minimal sufficient is that it can be written as a function of the likelihood ratio.

From this it becomes immediately clear that the likelihood ratio has to itself be minimal sufficient.

This also leads to the conclusion that:

If there exist $x_1, x_2 \in \mathcal{X}$ such that $f(x_1)=f(x_2)$ but $g(x_1) \not= g(x_2)$, then $g$ can not be written as a function of $f$, i.e. there exists no function $h$ with $g = h \circ f$.

Thus the condition isn't actually as difficult to show as I had thought.


*Keener doesn't address the issue of whether a statistic needs to be a measurable or just an arbitrary function or not. However, I am pretty sure that a statistic has to be a measurable function, because otherwise we couldn't define a distribution for it, i.e. a pullback measure.

**If $h$ were not measurable, we would have a contradiction because both $f$ and $g$ are measurable and the composition of measurable functions is again measurable. At the very least, $h$ has to be measurable restricted to $f(\mathcal{X}) \subseteq \mathcal{Y}$, although I think this would mean in most reasonable cases that $h$ would have to agree on $f(\mathcal{X})$ with a function that is measurable on all of $\mathcal{Y}$ (take $h|_{f(\mathcal{X})}$ on $f(\mathcal{X})$ and e.g. $z$ on $Y \setminus f(\mathcal{X})$ if there exists a measurable point $z \in \mathcal{Z}$, note that both $f(\mathcal{X})$ and $Y \setminus f(\mathcal{X})$ should be measurable in $Y$) so w.l.o.g. $h$ can be assumed to be measurable on all of $\mathcal{Y}$.

***At least this is necessary and sufficient for the existence of an arbitrary function factoring through $g$ and over $f$, and I think ** implies that if such an arbitrary function exists, this function also must be measurable, since both $f$ and $g$ are, i.e. it really would be a statistic $\mathcal{Y} \to \mathcal{Z}$.

****The condition given is equivalent to $T$ being sufficient by the factorization theorem, 3.6.

$\endgroup$
2
  • 1
    $\begingroup$ How do you define the likelihood ratio? $\endgroup$
    – Xi'an
    Commented Jun 4, 2018 at 6:44
  • $\begingroup$ @Xi'an I don't really remember all of the stupid stuff I wrote above, so to be honest I'm not sure which part you are referring to. If you are implicitly suggesting that I first prove that the likelihood ratio statistic is minimal sufficient, and then reduce any other proof of minimal sufficiency to suitable "sufficiency equivalence" with the likelihood ratio statistic, that probably is helpful in practice, but at least theoretically only seems to kick the can down the road (because then how does one understand the proof of minimal sufficiency of the LR statistic?) $\endgroup$ Commented Jun 11, 2018 at 1:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.