Let $X_1,...,X_n$ be an i.i.d. sample with parameter $\theta$ and $T$ a statistics. The statistics is called sufficient if, given a value $t$, the distribution $P_{\theta}(X_1,..,X_n|T=t)$ does not depend on $\theta$.

But what is the range of $t$ ? Do we need also that the event $P_{\theta}(T=t)>0$ for every value of $\theta$ in order to be sure that we are not conditioning on null events ? Do we need that $Supp_{\theta} f_T$ is the same for every value of $\theta$ ? Does it make sense this doubt ?

ps: the question is posed in the setting of discrete distribution.

  • $\begingroup$ As a counter-example, consider the Uniform distribution on $\{1,2,\ldots,\theta\}$ and the use of the sufficient statistic $T_n=\max_{1\le i\le n} X_i$. $\endgroup$ – Xi'an Oct 24 at 13:03
  • $\begingroup$ Exactly. In your second example, given $t$, $P_{\theta}(X_1,...,X_n|T=t)$ cannot be evaluated for every value of $\theta$: we need $\theta \ge t$ for the event $P_{\theta}(T=t)$ to have non null probability. Isn't there a problem there? What am I missing ? The statistics you proposed is supposed to be sufficient. $\endgroup$ – Thomas Oct 24 at 13:28

In advance probability texts, sufficiency is usually defined formally in terms of partitions on the sample space, and then we build up the standard definition as an implication of this for a parametric model. In any case, once you translate to the common definition, the requirement for sufficiency is that this condition should hold for all $t$. So a statistic $T: \mathbb{R} \rightarrow \Lambda$ will be sufficient for $\theta$ if and only if:

$$P_{\theta}(\mathbf{X}|T=t) = P_{\theta'}(\mathbf{X}|T=t) \quad \quad \quad \text{for all } \theta \neq \theta' \text{ and } t \in \Lambda.$$

If you would like to know more about the underlying definition, I recommend you have a look at the definition of a sufficient partition first. Most textbooks on intermediate or advanced probability will have an explanation of sufficiency that is couched in an initial definition in terms of a partition on the sample space.

  • $\begingroup$ Thanks. I see I think where you are going. But saying "a subspace that occurs with probability one" is a bit undefined no ? According to the value of $\theta$ we have different induced probabilities on $\Lambda$. $\endgroup$ – Thomas Oct 24 at 13:43
  • $\begingroup$ Ben, I do not understand the answer as it does not address the issue of the support of the statistic $T$ depending on the parameter $\theta$. This does not seem to relate with the statistic $T$ being defined up to a set of measure zero. $\endgroup$ – Xi'an Oct 27 at 8:15

There is no impact on sufficiency of the fact that $\mathbb{P}_\theta(T=t)$ is not always positive. As indicated in my example, the statistic $$T=X_{(n)}=\max_{1\le i\le n} X_i$$ is sufficient when the $X_i$'s are uniform on $\{1,\ldots,\theta\}$. And $$\mathbb{P}_\theta(T=t)=0$$ when $\theta<t$. The joint probability mass function of $(X_1,\ldots,X_n,X_{(n)})$ at $(x_1,\ldots,x_n,x_{(n)})$ is also zero when $x_{(n)}>\theta$, hence the conditional probability of $(X_1,\ldots,X_n)$ given $X_{(n)}=t$ and $t>\theta$ is not defined. But since the conditional distribution of $(x_1,\ldots,x_n)$ given $X_{(n)}=t$ is uniform over $$\{(x_1,\ldots,);\ x_{(n)}=t\}$$ independently of $\theta\ge t$ (and not defined otherwise), this does not impact sufficiency.

I presume that the undefined nature of the conditional in the impossible situation that $X_{(n)}=t$ and $t>\theta$ appears to bring some dependence on $\theta$ but this is not a correct impression: the conditional is not defined because the conditioning event is impossible. The part that brings information on $\theta$ is $X_{(n)}$, which is fine since it is sufficient.

Here is a quote from one of the earlier papers on the topic, by Koopman (1935):

enter image description here

where he similarly sees no impact on sufficiency in the fact that the density may be null for the actual observations and some values of the parameter.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.