2
$\begingroup$

In statistical model $(\mathcal{X}, \{P_\theta\mid\theta\in\Theta\})$ statistic $T=T(\mathbf{X})$ (where $\mathbf{X}$ marks random sample) is said to be sufficient for $\theta$, when conditional distribution of random sample with respect to $T$ does not depend on $\theta$. In modern probability approach conditional probability is defined in terms of condinal expectations, i.e. $$ P_\theta(A\mid\ T)=P_\theta(A\mid\sigma(T))\equiv \mathbb{E}_\theta[\mathbb{1}_A(\mathbf{X})\mid T],\quad\forall\theta\in\Theta$$where $A\in\mathcal{B}(\mathbb{R}^n)$. It means that $T$ is sufficient statistic if and only if conditional probability is a statistic (function of random sample, which does not depend on $\theta).$ If someone meet the definition like this somewhere? Maybe it's something wrong with it? I'm just curious.

$\endgroup$
0

1 Answer 1

2
$\begingroup$

If you're asking for a rigorous definition of sufficiency using modern probability theory/measure theory, then one of the following definitions (in decreasing levels of generality) might be what you're looking for.


Definition 1. (Sufficient sub-$\sigma$-algebra) Let $(\mathcal{X}, \mathcal{B})$ be a measurable space (the sample space), and let $\mathcal{P}$ be a set of probability measures on $(\mathcal{X}, \mathcal{B})$ (the candidate distributions). Let $\mathcal{S}$ be a sub-$\sigma$-algebra of $\mathcal{B}$. We say that $\mathcal{S}$ is sufficient for $\mathcal{P}$ if there exists a transition probability kernel $r : \mathcal{X} \times \mathcal{B} \to [0, 1]$ such that $r(\cdot, B)$ is a version of $P(B \mid \mathcal{S})$ for each $B \in \mathcal{B}$ and $P \in \mathcal{P}$.

That is, when conditioning on a sufficient $\sigma$-algebra $\mathcal{S}$, we no longer distinguish between the various candidate distributions $P \in \mathcal{P}$. This is the meaning of the line "$r(\cdot, B)$ is a version of $P(B \mid \mathcal{S})$ for each $B \in \mathcal{B}$ and $P \in \mathcal{P}$."

Unwrapping this definition a bit, $\mathcal{S}$ is sufficient for $\mathcal{P}$ if and only if there exists a function $r : \mathcal{X} \times \mathcal{B} \to [0, 1]$ such that the following all hold.

  1. For each $B \in \mathcal{B}$, the function $x \mapsto r(x, B)$ from $\mathcal{X}$ into $[0, 1]$ is $\mathcal{S}$-measurable.

  2. For each $x \in \mathcal{X}$, the function $B \mapsto r(x, B)$ from $\mathcal{B}$ into $[0, 1]$ is a probability measure on $(\mathcal{X}, \mathcal{B})$.

  3. For all $B \in \mathcal{B}$, $S \in \mathcal{S}$, and $P \in \mathcal{P}$, we have $$ P(B \cap S) = \int_S r(\cdot, B) \, dP. $$


Definition 2. (Sufficient statistic) Let $(\mathcal{X}, \mathcal{B})$ be a measurable space (the sample space), and let $\mathcal{P}$ be a set of probability measures on $(\mathcal{X}, \mathcal{B})$ (the candidate distributions). Let $(\mathcal{T}, \mathcal{C})$ be another measurable space. A statistic $T : \mathcal{X} \to \mathcal{T}$ is sufficient for $\mathcal{P}$ if the generated $\sigma$-algebra $$ \sigma(T) = \big\{\{T \in C\} : C \in \mathcal{C}\big\} $$ is sufficient for $\mathcal{T}$ in the sense of Definition 1.

We can also unpack Definition 2. A statistic $T$ is sufficient for $\mathcal{P}$ if and only if there exists a function $r : \mathcal{X} \times \mathcal{B} \to [0, 1]$ such that the following all hold.

  1. For each $B \in \mathcal{B}$, the function $x \mapsto r(x, B)$ from $\mathcal{X}$ into $[0, 1]$ is $\sigma(T)$-measurable.

  2. For each $x \in \mathcal{X}$, the function $B \mapsto r(x, B)$ from $\mathcal{B}$ into $[0, 1]$ is a probability measure on $(\mathcal{X}, \mathcal{B})$.

  3. For all $B \in \mathcal{B}$, $C \in \mathcal{C}$, and $P \in \mathcal{P}$, we have $$ P(B \cap \{T \in C\}) = \int_{\{T \in C\}} r(\cdot, B) \, dP. $$

In some cases (e.g., when $(\mathcal{X}, \mathcal{B})$ and $(\mathcal{T}, \mathcal{C})$ are standard Borel spaces, which includes essentially all spaces in practice), $T$ being sufficient for $\mathcal{P}$ is equivalent to the existence of a function $r^\prime : \mathcal{T} \times \mathcal{B} \to [0, 1]$ satisfying the following properties.

  1. For each $B \in \mathcal{B}$, the function $t \mapsto r^\prime(t, B)$ from $\mathcal{T}$ into $[0, 1]$ is $\mathcal{C}$-measurable.

  2. For each $t \in \mathcal{T}$, the function $B \mapsto r^\prime(t, B)$ from $\mathcal{B}$ into $[0, 1]$ is a probability measure on $(\mathcal{X}, \mathcal{B})$.

  3. For all $B \in \mathcal{B}$, $C \in \mathcal{C}$, and $P \in \mathcal{P}$, we have $$ P(B \cap \{T \in C\}) = \int_C r^\prime(\cdot, B) \, dT_*P, $$ where $T_*P$ is the distribution of $T$ under $P$: $(T_*P)(C) = P(\{T \in C\})$ for all $C \in \mathcal{C}$.

Heuristically, $r^\prime(t, B) = \mathbf{P}(X \in B \mid T = t)$, where $X$ is the "data" and $\mathbf{P}$ is some underlying, "ground truth" probability measure: it does not depend on the set $\mathcal{P}$ of candidate distributions.


Now consider the parametric case. A parametric statistical model consists of

  • a measurable space $(\mathcal{X}, \mathcal{B})$ (the sample space),
  • a measurable space $(\Theta, \tau)$ (the parameter space),
  • and a transition probability kernel $\Theta \times \mathcal{B} \to [0, 1]$, denoted $(\theta, B) \mapsto P_\theta(B)$ for $\theta \in \Theta$ and $B \in \mathcal{B}$, which is the parametrization of the sampling distribution.

In this setup, we can define the obvious family of candidate distributions $\mathcal{P} = \{P_\theta : \theta \in \Theta\}$, and then the two definitions of sufficiency given above carry over unchanged.

For example, a statistic $T : \mathcal{X} \to \mathcal{T}$ is sufficient for $\mathcal{P}$ (or by slight abuse of notation, for $P$ or $\theta$ or $P_\theta$) if and only if there exists a function $r : \mathcal{X} \times \mathcal{B} \to [0, 1]$ such that the following hold.

  1. For each $B \in \mathcal{B}$, the function $x \mapsto r(x, B)$ from $\mathcal{X}$ into $[0, 1]$ is $\sigma(T)$-measurable.

  2. For each $x \in \mathcal{X}$, the function $B \mapsto r(x, B)$ from $\mathcal{B}$ into $[0, 1]$ is a probability measure on $(\mathcal{X}, \mathcal{B})$.

  3. For all $B \in \mathcal{B}$, $C \in \mathcal{C}$, and $\theta \in \Theta$, we have $$ P_\theta(B \cap \{T \in C\}) = \int_{\{T \in C\}} r(\cdot, B) \, dP_\theta. $$

In particular, $P_\theta(B \mid T) = r(\cdot, B)$ for all $\theta \in \Theta$, so when we condition on $T$, all $P_\theta$'s become the same.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.