Conditioning in the definition of sufficient statistics

Question

Let $X_1,...,X_n$ be an i.i.d. sample with parameter $\theta$ and $T$ a statistics. The statistics is called sufficient if, given a value $t$, the distribution $P_{\theta}(X_1,..,X_n|T=t)$ does not depend on $\theta$.

But what is the range of $t$ ? Do we need also that the event $P_{\theta}(T=t)>0$ for every value of $\theta$ in order to be sure that we are not conditioning on null events ? Do we need that $Supp_{\theta} f_T$ is the same for every value of $\theta$ ? Does it make sense this doubt ?

ps: the question is posed in the setting of discrete distribution.

Answer 1

In advance probability texts, sufficiency is usually defined formally in terms of partitions on the sample space, and then we build up the standard definition as an implication of this for a parametric model. In any case, once you translate to the common definition, the requirement for sufficiency is that this condition should hold for all $t$. So a statistic $T: \mathbb{R} \rightarrow \Lambda$ will be sufficient for $\theta$ if and only if:

$$P_{\theta}(\mathbf{X}|T=t) = P_{\theta'}(\mathbf{X}|T=t) \quad \quad \quad \text{for all } \theta \neq \theta' \text{ and } t \in \Lambda.$$

If you would like to know more about the underlying definition, I recommend you have a look at the definition of a sufficient partition first. Most textbooks on intermediate or advanced probability will have an explanation of sufficiency that is couched in an initial definition in terms of a partition on the sample space.

Answer 2

There is no impact on sufficiency of the fact that $\mathbb{P}_\theta(T=t)$ is not always positive. As indicated in my example, the statistic $$T=X_{(n)}=\max_{1\le i\le n} X_i$$ is sufficient when the $X_i$'s are uniform on $\{1,\ldots,\theta\}$. And $$\mathbb{P}_\theta(T=t)=0$$ when $\theta<t$. The joint probability mass function of $(X_1,\ldots,X_n,X_{(n)})$ at $(x_1,\ldots,x_n,x_{(n)})$ is also zero when $x_{(n)}>\theta$, hence the conditional probability of $(X_1,\ldots,X_n)$ given $X_{(n)}=t$ and $t>\theta$ is not defined. But since the conditional distribution of $(x_1,\ldots,x_n)$ given $X_{(n)}=t$ is uniform over $$\{(x_1,\ldots,);\ x_{(n)}=t\}$$ independently of $\theta\ge t$ (and not defined otherwise), this does not impact sufficiency.

I presume that the undefined nature of the conditional in the impossible situation that $X_{(n)}=t$ and $t>\theta$ appears to bring some dependence on $\theta$ but this is not a correct impression: the conditional is not defined because the conditioning event is impossible. The part that brings information on $\theta$ is $X_{(n)}$, which is fine since it is sufficient.

Here is a quote from one of the earlier papers on the topic, by Koopman (1935):

where he similarly sees no impact on sufficiency in the fact that the density may be null for the actual observations and some values of the parameter.