# Conditional probability, statistic and sufficient statistic

In statistical model $$(\mathcal{X}, \{P_\theta\mid\theta\in\Theta\})$$ statistic $$T=T(\mathbf{X})$$ (where $$\mathbf{X}$$ marks random sample) is said to be sufficient for $$\theta$$, when conditional distribution of random sample with respect to $$T$$ does not depend on $$\theta$$. In modern probability approach conditional probability is defined in terms of condinal expectations, i.e. $$P_\theta(A\mid\ T)=P_\theta(A\mid\sigma(T))\equiv \mathbb{E}_\theta[\mathbb{1}_A(\mathbf{X})\mid T],\quad\forall\theta\in\Theta$$where $$A\in\mathcal{B}(\mathbb{R}^n)$$. It means that $$T$$ is sufficient statistic if and only if conditional probability is a statistic (function of random sample, which does not depend on $$\theta).$$ If someone meet the definition like this somewhere? Maybe it's something wrong with it? I'm just curious.

If you're asking for a rigorous definition of sufficiency using modern probability theory/measure theory, then one of the following definitions (in decreasing levels of generality) might be what you're looking for.

Definition 1. (Sufficient sub-$$\sigma$$-algebra) Let $$(\mathcal{X}, \mathcal{B})$$ be a measurable space (the sample space), and let $$\mathcal{P}$$ be a set of probability measures on $$(\mathcal{X}, \mathcal{B})$$ (the candidate distributions). Let $$\mathcal{S}$$ be a sub-$$\sigma$$-algebra of $$\mathcal{B}$$. We say that $$\mathcal{S}$$ is sufficient for $$\mathcal{P}$$ if there exists a transition probability kernel $$r : \mathcal{X} \times \mathcal{B} \to [0, 1]$$ such that $$r(\cdot, B)$$ is a version of $$P(B \mid \mathcal{S})$$ for each $$B \in \mathcal{B}$$ and $$P \in \mathcal{P}$$.

That is, when conditioning on a sufficient $$\sigma$$-algebra $$\mathcal{S}$$, we no longer distinguish between the various candidate distributions $$P \in \mathcal{P}$$. This is the meaning of the line "$$r(\cdot, B)$$ is a version of $$P(B \mid \mathcal{S})$$ for each $$B \in \mathcal{B}$$ and $$P \in \mathcal{P}$$."

Unwrapping this definition a bit, $$\mathcal{S}$$ is sufficient for $$\mathcal{P}$$ if and only if there exists a function $$r : \mathcal{X} \times \mathcal{B} \to [0, 1]$$ such that the following all hold.

1. For each $$B \in \mathcal{B}$$, the function $$x \mapsto r(x, B)$$ from $$\mathcal{X}$$ into $$[0, 1]$$ is $$\mathcal{S}$$-measurable.

2. For each $$x \in \mathcal{X}$$, the function $$B \mapsto r(x, B)$$ from $$\mathcal{B}$$ into $$[0, 1]$$ is a probability measure on $$(\mathcal{X}, \mathcal{B})$$.

3. For all $$B \in \mathcal{B}$$, $$S \in \mathcal{S}$$, and $$P \in \mathcal{P}$$, we have $$P(B \cap S) = \int_S r(\cdot, B) \, dP.$$

Definition 2. (Sufficient statistic) Let $$(\mathcal{X}, \mathcal{B})$$ be a measurable space (the sample space), and let $$\mathcal{P}$$ be a set of probability measures on $$(\mathcal{X}, \mathcal{B})$$ (the candidate distributions). Let $$(\mathcal{T}, \mathcal{C})$$ be another measurable space. A statistic $$T : \mathcal{X} \to \mathcal{T}$$ is sufficient for $$\mathcal{P}$$ if the generated $$\sigma$$-algebra $$\sigma(T) = \big\{\{T \in C\} : C \in \mathcal{C}\big\}$$ is sufficient for $$\mathcal{T}$$ in the sense of Definition 1.

We can also unpack Definition 2. A statistic $$T$$ is sufficient for $$\mathcal{P}$$ if and only if there exists a function $$r : \mathcal{X} \times \mathcal{B} \to [0, 1]$$ such that the following all hold.

1. For each $$B \in \mathcal{B}$$, the function $$x \mapsto r(x, B)$$ from $$\mathcal{X}$$ into $$[0, 1]$$ is $$\sigma(T)$$-measurable.

2. For each $$x \in \mathcal{X}$$, the function $$B \mapsto r(x, B)$$ from $$\mathcal{B}$$ into $$[0, 1]$$ is a probability measure on $$(\mathcal{X}, \mathcal{B})$$.

3. For all $$B \in \mathcal{B}$$, $$C \in \mathcal{C}$$, and $$P \in \mathcal{P}$$, we have $$P(B \cap \{T \in C\}) = \int_{\{T \in C\}} r(\cdot, B) \, dP.$$

In some cases (e.g., when $$(\mathcal{X}, \mathcal{B})$$ and $$(\mathcal{T}, \mathcal{C})$$ are standard Borel spaces, which includes essentially all spaces in practice), $$T$$ being sufficient for $$\mathcal{P}$$ is equivalent to the existence of a function $$r^\prime : \mathcal{T} \times \mathcal{B} \to [0, 1]$$ satisfying the following properties.

1. For each $$B \in \mathcal{B}$$, the function $$t \mapsto r^\prime(t, B)$$ from $$\mathcal{T}$$ into $$[0, 1]$$ is $$\mathcal{C}$$-measurable.

2. For each $$t \in \mathcal{T}$$, the function $$B \mapsto r^\prime(t, B)$$ from $$\mathcal{B}$$ into $$[0, 1]$$ is a probability measure on $$(\mathcal{X}, \mathcal{B})$$.

3. For all $$B \in \mathcal{B}$$, $$C \in \mathcal{C}$$, and $$P \in \mathcal{P}$$, we have $$P(B \cap \{T \in C\}) = \int_C r^\prime(\cdot, B) \, dT_*P,$$ where $$T_*P$$ is the distribution of $$T$$ under $$P$$: $$(T_*P)(C) = P(\{T \in C\})$$ for all $$C \in \mathcal{C}$$.

Heuristically, $$r^\prime(t, B) = \mathbf{P}(X \in B \mid T = t)$$, where $$X$$ is the "data" and $$\mathbf{P}$$ is some underlying, "ground truth" probability measure: it does not depend on the set $$\mathcal{P}$$ of candidate distributions.

Now consider the parametric case. A parametric statistical model consists of

• a measurable space $$(\mathcal{X}, \mathcal{B})$$ (the sample space),
• a measurable space $$(\Theta, \tau)$$ (the parameter space),
• and a transition probability kernel $$\Theta \times \mathcal{B} \to [0, 1]$$, denoted $$(\theta, B) \mapsto P_\theta(B)$$ for $$\theta \in \Theta$$ and $$B \in \mathcal{B}$$, which is the parametrization of the sampling distribution.

In this setup, we can define the obvious family of candidate distributions $$\mathcal{P} = \{P_\theta : \theta \in \Theta\}$$, and then the two definitions of sufficiency given above carry over unchanged.

For example, a statistic $$T : \mathcal{X} \to \mathcal{T}$$ is sufficient for $$\mathcal{P}$$ (or by slight abuse of notation, for $$P$$ or $$\theta$$ or $$P_\theta$$) if and only if there exists a function $$r : \mathcal{X} \times \mathcal{B} \to [0, 1]$$ such that the following hold.

1. For each $$B \in \mathcal{B}$$, the function $$x \mapsto r(x, B)$$ from $$\mathcal{X}$$ into $$[0, 1]$$ is $$\sigma(T)$$-measurable.

2. For each $$x \in \mathcal{X}$$, the function $$B \mapsto r(x, B)$$ from $$\mathcal{B}$$ into $$[0, 1]$$ is a probability measure on $$(\mathcal{X}, \mathcal{B})$$.

3. For all $$B \in \mathcal{B}$$, $$C \in \mathcal{C}$$, and $$\theta \in \Theta$$, we have $$P_\theta(B \cap \{T \in C\}) = \int_{\{T \in C\}} r(\cdot, B) \, dP_\theta.$$

In particular, $$P_\theta(B \mid T) = r(\cdot, B)$$ for all $$\theta \in \Theta$$, so when we condition on $$T$$, all $$P_\theta$$'s become the same.