# What is the intuition behind the factorization theorem? (Sufficient statistics)

By the Fisher's factorization theorem, a statistics is a sufficient statistic if (and only if) the joint density,

$$f(x_1, x_2, x_3, \dots x_n; \theta)$$

can be factorized into two functions, $$g(s; \theta)$$ and $$h(x_1, x_2, \dots x_n)$$.

What is the intuition behind this theorem?

I know that a proof exists for this theorem, and I also understand the original definition for a sufficient statistic (that the conditional distribution of the sample, given the statistic should not depend on the parameter).

• But then, to find the actual probability $f(x_1, x_2, x_3, ... x_n; \theta)$, we'd still need the proportionality constant right? Commented Nov 15, 2019 at 8:09

Fisher's factorisation theorem is one of several ways to establish or prove that a statistic $$S_n(X_1,\ldots,X_n)$$ is sufficient. The meaning of sufficiency remains identical through all these manners of characterising it though, namely that the conditional distribution of the sample $$X_1,\ldots,X_n$$ conditional on $$S_n(X_1,\ldots,X_n)$$ is constant in $$\theta$$, i.e., is the same distribution for all values of $$\theta$$.
For instance, given an iid sample $$X_1,\ldots,X_n$$, the order statistic $$S_n(X_1,\ldots,X_n)=(X_{(1)},\ldots,X_{(n)})$$ is sufficient because the distribution of $$X_1,\ldots,X_n$$ given $$S_n(X_1,\ldots,X_n)$$ is the uniform distribution on the permutations of $$\{1,\ldots,n\}$$: \begin{align*}\mathbb P_\theta((X_1,\ldots,X_n)&=(y_1,\ldots,y_n)|S_n(X_1,\ldots,X_n)=(x_{(1)},\ldots,x_{(n)})\\&=\frac{\mathbb I_{(x_{(1)},\ldots,x_{(n)})}(y_{(1)},\ldots,y_{(n)})}{n!}\end{align*}
The factorisation theorem thus does not modify the original distribution of the sample and does not particularly help in finding it. It operates the other way, as a mean of figuring out sufficient statistics by looking at the joint pmf or pdf. Once a sufficient statistic has been found, its own distribution is sufficient (in the sense of "enough") for creating a likelihood function in the parameter $$\theta$$. It will then differ from the "full" likelihood by a constant (in the parameter) equal to the function (of the sample) $$h(x_1,\ldots,x_n)$$ found in the factorisation theorem. But this constant has no relevance for statistical inference.
• I agree with @Xi'an, here is a link talking about the factorization theorem. 'it is not always all that easy to find the conditional distribution of $X_1, X_2, ..., X_n$ given $Y$. Not to mention that we'd have to find the conditional distribution of $X_1, X_2, ..., X_n$ given $Y$ for every $Y$ that we'd want to consider a possible sufficient statistic!' newonlinecourses.science.psu.edu/stat414/node/283 Commented Nov 14, 2019 at 17:15
The factorization theorem states that $$S(X)$$ is sufficient if and only if the likelihood function of $$\theta$$ for data $$X$$ can be factored into the product of a function of $$X$$ (constant in $$\theta$$) and a function of $$S(X)$$ and $$\theta$$.
As a consequence, the likelihood functions of $$\theta$$ with respect to $$X$$ and $$S(X)$$ are proportional to each other, for fixed $$X$$. But proportional likelihood functions are equivalent in the sense that likelihood ratios of $$\theta_1$$ and $$\theta_2$$ are the same. Therefore, processing or reducing the data $$X$$ by a sufficient statistic $$S$$ will not change your inferences about $$\theta$$ (if you're doing likelihood-based inferences, of course), as it keeps all the information you'll need to infer about the parameter $$\theta$$.