4
$\begingroup$

By the Fisher's factorization theorem, a statistics is a sufficient statistic if (and only if) the joint density,

$$ f(x_1, x_2, x_3, \dots x_n; \theta) $$

can be factorized into two functions, $ g(s; \theta) $ and $ h(x_1, x_2, \dots x_n) $. What is the intutition behind this theorem. I know that a proof exists for this theorem, and I also understand the original definition for a sufficient statistic (that the condition distribution of the sample, given the statistic should not depend on the parameter).

$\endgroup$
1
  • $\begingroup$ But then, to find the actual probability $ f(x_1, x_2, x_3, ... x_n; \theta) $, we'd still need the proportionality constant right? $\endgroup$
    – WorldGov
    Nov 15, 2019 at 8:09

1 Answer 1

2
$\begingroup$

Fisher's factorisation theorem is one of several ways to establish or prove that a statistic $S_n(X_1,\ldots,X_n)$ is sufficient. The meaning of sufficiency remains identical through all these manners of characterising it though, namely that the conditional distribution of the sample $X_1,\ldots,X_n$ conditional on $S_n(X_1,\ldots,X_n)$ is constant in $\theta$, i.e., is the same distribution for all values of $\theta$.

For instance, given an iid sample $X_1,\ldots,X_n$, the order statistic $S_n(X_1,\ldots,X_n)=(X_{(1)},\ldots,X_{(n)})$ is sufficient because the distribution of $X_1,\ldots,X_n$ given $S_n(X_1,\ldots,X_n)$ is the uniform distribution on the permutations of $\{1,\ldots,n\}$: \begin{align*}\mathbb P_\theta((X_1,\ldots,X_n)&=(y_1,\ldots,y_n)|S_n(X_1,\ldots,X_n)=(x_{(1)},\ldots,x_{(n)})\\&=\frac{\mathbb I_{(x_{(1)},\ldots,x_{(n)})}(y_{(1)},\ldots,y_{(n)})}{n!}\end{align*}

enter image description hereThe factorisation theorem thus does not modify the original distribution of the sample and does not particularly help in finding it. It operates the other way, as a mean of figuring out sufficient statistics by looking at the joint pmf or pdf. Once a sufficient statistic has been found, its own distribution is sufficient (in the sense of "enough") for creating a likelihood function in the parameter $\theta$. It will then differ from the "full" likelihood by a constant (in the parameter) equal to the function (of the sample) $h(x_1,\ldots,x_n)$ found in the factorisation theorem. But this constant has no relevance for statistical inference.

$\endgroup$
1
  • $\begingroup$ I agree with @Xi'an, here is a link talking about the factorization theorem. 'it is not always all that easy to find the conditional distribution of $X_1, X_2, ..., X_n$ given $Y$. Not to mention that we'd have to find the conditional distribution of $X_1, X_2, ..., X_n$ given $Y$ for every $Y$ that we'd want to consider a possible sufficient statistic!' newonlinecourses.science.psu.edu/stat414/node/283 $\endgroup$
    – Bill Chen
    Nov 14, 2019 at 17:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.