My notes introduce the concept of minimal sufficient statistics as follows:


A sufficient statistic $T(\mathbf{Y})$ is called a minimal sufficient statistic if it is a function of any other sufficient statistic.


Except for several very special examples, a minimal sufficient statistic always exists.

Assume the existence of a minimal sufficient statistic and consider partitioning the sample space $\Omega$, where $\mathbf{y}_1, \mathbf{y}_2 \in \Omega$ are assigned to the same equivalence class iff the likelihood ratio $L(\theta; \mathbf{y})/L(\theta,\mathbf{y})$ does not depend on $\theta$.

Define a statistic $T(\mathbf{Y})$ in such a way that $T(\mathbf{y}_1) = T(\mathbf{y_2})$ if $\mathbf{y}_1$ and $\mathbf{y}_2$ belong to the same equivalence class and $T(\mathbf{y}_1) \not= T(\mathbf{y_2})$ otherwise.

Therorem 2

The statistic $T(\mathbf{Y})$ defined above is the minimal sufficient statistic for $\theta$.

Proof of theorem 2 for the discrete case

First we show that $T(\mathbf{Y})$ is sufficient.

$$\begin{align} P_\theta (\mathbf{Y} = \mathbf{y} \vert T(\mathbf{Y}) = t) &= \dfrac{P(\mathbf{Y} = \mathbf{y}, T(\mathbf{Y}) = t)}{P(T(\mathbf{Y}) = t)} \\ &= \dfrac{P(\mathbf{Y} = \mathbf{y})}{\sum_{\mathbf{y}_i : T(\mathbf{y}_i) = t} P(\mathbf{Y} = \mathbf{y}_i)} \\ &= \dfrac{L(\theta; \mathbf{y})}{\sum_{\mathbf{y}_i : T(\mathbf{y}_i) = t} L(\theta; \mathbf{y})} \\ &= \dfrac{1}{\sum_{\mathbf{y}_i : T(\mathbf{y}_i) = t} \dfrac{L(\theta; \mathbf{y}_i)}{L(\theta; \mathbf{y})}} \end{align}$$

Since $T(\mathbf{y}) = T(\mathbf{y}_i) = t$, all $\mathbf{y}_i$ and $\mathbf{y}$ belong to the same equivalence class induced by $T(\mathbf{y})$ and, therefore, the likelihood ratios $L(\theta; \mathbf{y}_i)/L(\theta; \mathbf{y})$ do not depend on $\theta$.

What is the reasoning that leads to the changes in the numerator and denominator of $\dfrac{P(\mathbf{Y} = \mathbf{y}, T(\mathbf{Y}) = t)}{P(T(\mathbf{Y}) = t)} = \dfrac{P(\mathbf{Y} = \mathbf{y})}{\sum_{\mathbf{y}_i : T(\mathbf{y}_i) = t} P(\mathbf{Y} = \mathbf{y}_i)}$? In the numerator, the joint PMF becomes a univariate PMF, and in the denominator, the PMF is changed from $P(T(\mathbf{Y}) = t)$ to $P(\mathbf{Y} = \mathbf{y}_i)$ and a sum is inserted.

I would greatly appreciate it if people would please take the time to explain this.


There is an assumption that $t=T(y)$, otherwise the equation is not true.

To see that the assumption is needed, suppose $t \ne T(y)$, then $P_\theta(Y=y,T(Y)=t)=0$ but $P(Y=y)$ can be positive.

$$P(Y=y, T(Y)=t)=P(Y=y)$$ since we already know that $T(y)=t$.

Also $$P(T(Y)=t)=\sum_{y_i:T(y_i)=t}P(Y=y_i)$$

since \begin{align}\{\omega: T(Y(\omega))=t\}&=\bigsqcup_{y_i:T(y_i)=t}\{\omega: Y(\omega)=y_i , T(Y(\omega))=t\} \\ &=\bigsqcup_{y_i:T(y_i)=t}\{\omega: Y(\omega)=y_i \}\end{align} the right hand side forms a partition for the set on the left and we used the law of total probability.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.