5
$\begingroup$

By the Fisher's factorization theorem, a statistics is a sufficient statistic if (and only if) the joint density,

$$ f(x_1, x_2, x_3, \dots x_n; \theta) $$

can be factorized into two functions, $ g(s; \theta) $ and $ h(x_1, x_2, \dots x_n) $.

What is the intuition behind this theorem?

I know that a proof exists for this theorem, and I also understand the original definition for a sufficient statistic (that the conditional distribution of the sample, given the statistic should not depend on the parameter).

$\endgroup$
1
  • $\begingroup$ But then, to find the actual probability $ f(x_1, x_2, x_3, ... x_n; \theta) $, we'd still need the proportionality constant right? $\endgroup$
    – WorldGov
    Commented Nov 15, 2019 at 8:09

2 Answers 2

3
$\begingroup$

Fisher's factorisation theorem is one of several ways to establish or prove that a statistic $S_n(X_1,\ldots,X_n)$ is sufficient. The meaning of sufficiency remains identical through all these manners of characterising it though, namely that the conditional distribution of the sample $X_1,\ldots,X_n$ conditional on $S_n(X_1,\ldots,X_n)$ is constant in $\theta$, i.e., is the same distribution for all values of $\theta$.

For instance, given an iid sample $X_1,\ldots,X_n$, the order statistic $S_n(X_1,\ldots,X_n)=(X_{(1)},\ldots,X_{(n)})$ is sufficient because the distribution of $X_1,\ldots,X_n$ given $S_n(X_1,\ldots,X_n)$ is the uniform distribution on the permutations of $\{1,\ldots,n\}$: \begin{align*}\mathbb P_\theta((X_1,\ldots,X_n)&=(y_1,\ldots,y_n)|S_n(X_1,\ldots,X_n)=(x_{(1)},\ldots,x_{(n)})\\&=\frac{\mathbb I_{(x_{(1)},\ldots,x_{(n)})}(y_{(1)},\ldots,y_{(n)})}{n!}\end{align*}

enter image description hereThe factorisation theorem thus does not modify the original distribution of the sample and does not particularly help in finding it. It operates the other way, as a mean of figuring out sufficient statistics by looking at the joint pmf or pdf. Once a sufficient statistic has been found, its own distribution is sufficient (in the sense of "enough") for creating a likelihood function in the parameter $\theta$. It will then differ from the "full" likelihood by a constant (in the parameter) equal to the function (of the sample) $h(x_1,\ldots,x_n)$ found in the factorisation theorem. But this constant has no relevance for statistical inference.

$\endgroup$
1
  • 1
    $\begingroup$ I agree with @Xi'an, here is a link talking about the factorization theorem. 'it is not always all that easy to find the conditional distribution of $X_1, X_2, ..., X_n$ given $Y$. Not to mention that we'd have to find the conditional distribution of $X_1, X_2, ..., X_n$ given $Y$ for every $Y$ that we'd want to consider a possible sufficient statistic!' newonlinecourses.science.psu.edu/stat414/node/283 $\endgroup$
    – Bill Chen
    Commented Nov 14, 2019 at 17:15
0
$\begingroup$

The factorization theorem states that $S(X)$ is sufficient if and only if the likelihood function of $\theta$ for data $X$ can be factored into the product of a function of $X$ (constant in $\theta$) and a function of $S(X)$ and $\theta$.

As a consequence, the likelihood functions of $\theta$ with respect to $X$ and $S(X)$ are proportional to each other, for fixed $X$. But proportional likelihood functions are equivalent in the sense that likelihood ratios of $\theta_1$ and $\theta_2$ are the same. Therefore, processing or reducing the data $X$ by a sufficient statistic $S$ will not change your inferences about $\theta$ (if you're doing likelihood-based inferences, of course), as it keeps all the information you'll need to infer about the parameter $\theta$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.