Sufficient statistic definition in Koller's Probabilistic Graphical Models

Question

In Daphne Koller's Probabilistic Graphical Models, the sufficient statistic is defined as follows (p 721):

A function $\tau(\xi)$ from instances of $\chi$ to $\mathbb R^l$ (for some $l$) is a sufficient statistic if, for any two data sets $\mathcal D$ and $\mathcal D'$ and any $\theta \in \Theta$, we have that: $$ \sum_{\xi[m] \in \mathcal D} \tau(\xi[m]) = \sum_{\xi[m] \in \mathcal D'} \tau(\xi[m]) \implies L(\theta: \mathcal D) = L(\theta: \mathcal D') $$

We often refer to the tuple $\sum_{\xi[m] \in \mathcal D} \tau(\xi[m])$ as the sufficient statistics of the data set $\mathcal D$.

Later, it gives an example that for the Gaussian distribution, $\tau(x)=<1, x, x^2>$ and so for sample $x_1, \dots, x_M$, the sufficient statistic for the data set is$$\sum_m \tau(x_m)=<M, \sum_m x, \sum_m x^2 > $$

The definition of sufficient statistic that I find in other sources seems different. For example, in Casella and Berger's Statistical Inference, it is defined as follows:

A statistic $T(X)$ is a sufficient statistic for $\theta$ if the conditional distribution of the sample $X$ given the value of $T(X)$ does not depend on $\theta$.

Note: the sample $X$ in the above is a vector consisting of $n$ data points drawn from some distribution.

I have three specific questions about the definition in the Probabilistic Graphical Model:

1. As far as I know, a sample of $m$ data points is a vector $(\xi[1], \xi[2], \dots, \xi[m])$, and a statistic on a sample is a function that maps this vector to $\mathbb R^l$. A sufficient statistic is a statistic, and so should be a mapping from $(\xi[1], \xi[2], \dots, \xi[M])$ to $\mathbb R^l$, like in the second definition. But in the PGM definition, it is a mapping $\tau$ from a single data point $\xi[m]$ to $\mathbb R^l$.

2. It later defined $\sum_{\xi[m] \in \mathcal D} \tau(\xi[m])$ as the sufficient statistics (plural) of the data set $\mathcal D$. This certainly is equivalent to the $T(X)$ in the second definition. But I am not sure the difference between a sufficient statistic (for the model) and sufficient statistics of data set $\mathcal D$.

3. The definition seems to be referring to a general case that is true for all distributions. Then I am not sure in $\sum_{\xi[m] \in \mathcal D} \tau(\xi[m])$, why $\tau(\xi[m])$ for each data point can be summed. I feel this can only be true for the exponential family.

I would appreciate some clarifications about this concept.

Answer 1

This definition of sufficiency by Koller & Friedman is both restrictive and incorrect:

1. A sufficient statistic for a sample of size $M$ is not always the sum of statistics of the elements of the sample. A counter-example is the Uniform $\mathcal U(0,\theta)$ distribution where a sufficient statistic is $\max(\xi_1,\ldots,\xi_M)$
2. When two samples of the same size return the same value of a sufficient statistic, their respective likelihood functions are proportional and not equal. This is a consequence of the factorisation theorem and is exemplified by the Normal $\mathcal N(\theta,1)$ model.
3. This definition excludes non-linear bijective transforms of sufficient statistics as being sufficient. Even for exponential families, it forces the sufficient statistic to be based on the natural statistics, i.e., $\tau:\mathfrak X\mapsto\mathbb R^\ell$ such that the density of the sample points writes as $p(x|\theta)\propto \exp\{\theta^\text{T}\tau(x)\}$ [wrt to a fixed measure].
4. Although this is not incorrect, the sample size $M$ has no place in a sufficient statistic as it is purely ancillary and non-random in the examples covered in that section.

As for the statistic-versus-statistics (minor) incoherence, it sounds to me that a statistic is first defined as a vector in $\mathbb R^\ell$ and then the "tuple" [which should be an $\ell$-uple!] is a vector made of components that are also statistics, hence a "tuple of sufficient statistics" [which would read better as a sufficient tuple of statistics!]