# Conditioning in the definition of sufficient statistics

Let $$X_1,...,X_n$$ be an i.i.d. sample with parameter $$\theta$$ and $$T$$ a statistics. The statistics is called sufficient if, given a value $$t$$, the distribution $$P_{\theta}(X_1,..,X_n|T=t)$$ does not depend on $$\theta$$.

But what is the range of $$t$$ ? Do we need also that the event $$P_{\theta}(T=t)>0$$ for every value of $$\theta$$ in order to be sure that we are not conditioning on null events ? Do we need that $$Supp_{\theta} f_T$$ is the same for every value of $$\theta$$ ? Does it make sense this doubt ?

ps: the question is posed in the setting of discrete distribution.

• As a counter-example, consider the Uniform distribution on $\{1,2,\ldots,\theta\}$ and the use of the sufficient statistic $T_n=\max_{1\le i\le n} X_i$. – Xi'an Oct 24 at 13:03
• Exactly. In your second example, given $t$, $P_{\theta}(X_1,...,X_n|T=t)$ cannot be evaluated for every value of $\theta$: we need $\theta \ge t$ for the event $P_{\theta}(T=t)$ to have non null probability. Isn't there a problem there? What am I missing ? The statistics you proposed is supposed to be sufficient. – Thomas Oct 24 at 13:28

In advance probability texts, sufficiency is usually defined formally in terms of partitions on the sample space, and then we build up the standard definition as an implication of this for a parametric model. In any case, once you translate to the common definition, the requirement for sufficiency is that this condition should hold for all $$t$$. So a statistic $$T: \mathbb{R} \rightarrow \Lambda$$ will be sufficient for $$\theta$$ if and only if:

$$P_{\theta}(\mathbf{X}|T=t) = P_{\theta'}(\mathbf{X}|T=t) \quad \quad \quad \text{for all } \theta \neq \theta' \text{ and } t \in \Lambda.$$

If you would like to know more about the underlying definition, I recommend you have a look at the definition of a sufficient partition first. Most textbooks on intermediate or advanced probability will have an explanation of sufficiency that is couched in an initial definition in terms of a partition on the sample space.

• Thanks. I see I think where you are going. But saying "a subspace that occurs with probability one" is a bit undefined no ? According to the value of $\theta$ we have different induced probabilities on $\Lambda$. – Thomas Oct 24 at 13:43
• Ben, I do not understand the answer as it does not address the issue of the support of the statistic $T$ depending on the parameter $\theta$. This does not seem to relate with the statistic $T$ being defined up to a set of measure zero. – Xi'an Oct 27 at 8:15

There is no impact on sufficiency of the fact that $$\mathbb{P}_\theta(T=t)$$ is not always positive. As indicated in my example, the statistic $$T=X_{(n)}=\max_{1\le i\le n} X_i$$ is sufficient when the $$X_i$$'s are uniform on $$\{1,\ldots,\theta\}$$. And $$\mathbb{P}_\theta(T=t)=0$$ when $$\theta. The joint probability mass function of $$(X_1,\ldots,X_n,X_{(n)})$$ at $$(x_1,\ldots,x_n,x_{(n)})$$ is also zero when $$x_{(n)}>\theta$$, hence the conditional probability of $$(X_1,\ldots,X_n)$$ given $$X_{(n)}=t$$ and $$t>\theta$$ is not defined. But since the conditional distribution of $$(x_1,\ldots,x_n)$$ given $$X_{(n)}=t$$ is uniform over $$\{(x_1,\ldots,);\ x_{(n)}=t\}$$ independently of $$\theta\ge t$$ (and not defined otherwise), this does not impact sufficiency.

I presume that the undefined nature of the conditional in the impossible situation that $$X_{(n)}=t$$ and $$t>\theta$$ appears to bring some dependence on $$\theta$$ but this is not a correct impression: the conditional is not defined because the conditioning event is impossible. The part that brings information on $$\theta$$ is $$X_{(n)}$$, which is fine since it is sufficient.

Here is a quote from one of the earlier papers on the topic, by Koopman (1935): where he similarly sees no impact on sufficiency in the fact that the density may be null for the actual observations and some values of the parameter.