# Joint distribution simplification in minimal sufficient statistics proof

My notes introduce the concept of minimal sufficient statistics as follows:

Definition

A sufficient statistic $$T(\mathbf{Y})$$ is called a minimal sufficient statistic if it is a function of any other sufficient statistic.

Remark

Except for several very special examples, a minimal sufficient statistic always exists.

Assume the existence of a minimal sufficient statistic and consider partitioning the sample space $$\Omega$$, where $$\mathbf{y}_1, \mathbf{y}_2 \in \Omega$$ are assigned to the same equivalence class iff the likelihood ratio $$L(\theta; \mathbf{y})/L(\theta,\mathbf{y})$$ does not depend on $$\theta$$.

Define a statistic $$T(\mathbf{Y})$$ in such a way that $$T(\mathbf{y}_1) = T(\mathbf{y_2})$$ if $$\mathbf{y}_1$$ and $$\mathbf{y}_2$$ belong to the same equivalence class and $$T(\mathbf{y}_1) \not= T(\mathbf{y_2})$$ otherwise.

Therorem 2

The statistic $$T(\mathbf{Y})$$ defined above is the minimal sufficient statistic for $$\theta$$.

Proof of theorem 2 for the discrete case

First we show that $$T(\mathbf{Y})$$ is sufficient.

\begin{align} P_\theta (\mathbf{Y} = \mathbf{y} \vert T(\mathbf{Y}) = t) &= \dfrac{P(\mathbf{Y} = \mathbf{y}, T(\mathbf{Y}) = t)}{P(T(\mathbf{Y}) = t)} \\ &= \dfrac{P(\mathbf{Y} = \mathbf{y})}{\sum_{\mathbf{y}_i : T(\mathbf{y}_i) = t} P(\mathbf{Y} = \mathbf{y}_i)} \\ &= \dfrac{L(\theta; \mathbf{y})}{\sum_{\mathbf{y}_i : T(\mathbf{y}_i) = t} L(\theta; \mathbf{y})} \\ &= \dfrac{1}{\sum_{\mathbf{y}_i : T(\mathbf{y}_i) = t} \dfrac{L(\theta; \mathbf{y}_i)}{L(\theta; \mathbf{y})}} \end{align}

Since $$T(\mathbf{y}) = T(\mathbf{y}_i) = t$$, all $$\mathbf{y}_i$$ and $$\mathbf{y}$$ belong to the same equivalence class induced by $$T(\mathbf{y})$$ and, therefore, the likelihood ratios $$L(\theta; \mathbf{y}_i)/L(\theta; \mathbf{y})$$ do not depend on $$\theta$$.

What is the reasoning that leads to the changes in the numerator and denominator of $$\dfrac{P(\mathbf{Y} = \mathbf{y}, T(\mathbf{Y}) = t)}{P(T(\mathbf{Y}) = t)} = \dfrac{P(\mathbf{Y} = \mathbf{y})}{\sum_{\mathbf{y}_i : T(\mathbf{y}_i) = t} P(\mathbf{Y} = \mathbf{y}_i)}$$? In the numerator, the joint PMF becomes a univariate PMF, and in the denominator, the PMF is changed from $$P(T(\mathbf{Y}) = t)$$ to $$P(\mathbf{Y} = \mathbf{y}_i)$$ and a sum is inserted.

There is an assumption that $$t=T(y)$$, otherwise the equation is not true.
To see that the assumption is needed, suppose $$t \ne T(y)$$, then $$P_\theta(Y=y,T(Y)=t)=0$$ but $$P(Y=y)$$ can be positive.
$$P(Y=y, T(Y)=t)=P(Y=y)$$ since we already know that $$T(y)=t$$.
Also $$P(T(Y)=t)=\sum_{y_i:T(y_i)=t}P(Y=y_i)$$
since \begin{align}\{\omega: T(Y(\omega))=t\}&=\bigsqcup_{y_i:T(y_i)=t}\{\omega: Y(\omega)=y_i , T(Y(\omega))=t\} \\ &=\bigsqcup_{y_i:T(y_i)=t}\{\omega: Y(\omega)=y_i \}\end{align} the right hand side forms a partition for the set on the left and we used the law of total probability.