# What is the intuition behind the factorization theorem? (Sufficient statistics)

By the Fisher's factorization theorem, a statistics is a sufficient statistic if (and only if) the joint density,

$$f(x_1, x_2, x_3, \dots x_n; \theta)$$

can be factorized into two functions, $$g(s; \theta)$$ and $$h(x_1, x_2, \dots x_n)$$. What is the intutition behind this theorem. I know that a proof exists for this theorem, and I also understand the original definition for a sufficient statistic (that the condition distribution of the sample, given the statistic should not depend on the parameter).

• But then, to find the actual probability $f(x_1, x_2, x_3, ... x_n; \theta)$, we'd still need the proportionality constant right? – WorldGov Nov 15 '19 at 8:09

## 1 Answer

Fisher's factorisation theorem is one of several ways to establish or prove that a statistic $$S_n(X_1,\ldots,X_n)$$ is sufficient. The meaning of sufficiency remains identical through all these manners of characterising it though, namely that the conditional distribution of the sample $$X_1,\ldots,X_n$$ conditional on $$S_n(X_1,\ldots,X_n)$$ is constant in $$\theta$$, i.e., is the same distribution for all values of $$\theta$$.

For instance, given an iid sample $$X_1,\ldots,X_n$$, the order statistic $$S_n(X_1,\ldots,X_n)=(X_{(1)},\ldots,X_{(n)})$$ is sufficient because the distribution of $$X_1,\ldots,X_n$$ given $$S_n(X_1,\ldots,X_n)$$ is the uniform distribution on the permutations of $$\{1,\ldots,n\}$$: \begin{align*}\mathbb P_\theta((X_1,\ldots,X_n)&=(y_1,\ldots,y_n)|S_n(X_1,\ldots,X_n)=(x_{(1)},\ldots,x_{(n)})\\&=\frac{\mathbb I_{(x_{(1)},\ldots,x_{(n)})}(y_{(1)},\ldots,y_{(n)})}{n!}\end{align*} The factorisation theorem thus does not modify the original distribution of the sample and does not particularly help in finding it. It operates the other way, as a mean of figuring out sufficient statistics by looking at the joint pmf or pdf. Once a sufficient statistic has been found, its own distribution is sufficient (in the sense of "enough") for creating a likelihood function in the parameter $$\theta$$. It will then differ from the "full" likelihood by a constant (in the parameter) equal to the function (of the sample) $$h(x_1,\ldots,x_n)$$ found in the factorisation theorem. But this constant has no relevance for statistical inference.

• I agree with @Xi'an, here is a link talking about the factorization theorem. 'it is not always all that easy to find the conditional distribution of $X_1, X_2, ..., X_n$ given $Y$. Not to mention that we'd have to find the conditional distribution of $X_1, X_2, ..., X_n$ given $Y$ for every $Y$ that we'd want to consider a possible sufficient statistic!' newonlinecourses.science.psu.edu/stat414/node/283 – Bill Chen Nov 14 '19 at 17:15