Suppose that $X_1, \ldots, X_n$ are iid Bernoulli trials with parameter $p$, and that $Y = \sum_{i=1}^nX_i$. Then, we have that $Y$ is a sufficient statistic by the following condensed proof:

$$ P(X_1 = x_1, ... , X_n = x_n |Y = y) = \frac{p^y(1-p)^{n-y}}{\binom{n}{y} p^y(1-p)^{n-y}} =\frac{1}{\binom{n}{y}} \text{ if } \sum_{i=1}^{n}x_i = y $$

I am trying to see why this is the case. My reasoning is that there is some cancellation property on top and bottom that "kills" off the parameter $p$. In that whatever is on top through $p$ is perfectly matched by whatever is on the bottom minus scaling factors (in this case $\binom{n}{y}^{-1}$).

However, I am wondering what the deeper meaning behind this is, that the numerator of the joint distribution of the data and the sufficient statistic has its EXACT form on the bottom, except for constants which are only multiplicative (to facilitate cancellations). Am I onto something bigger and deeper here?


Let $y$ be a fixed integer in $\{0, 1, 2, \ldots n\}$ and note that the event $(Y = y)$ is the union of $\binom{n}{y}$ disjoint equiprobable events of the form $(X_1 = x_1, X_2 = x_2, \ldots, X_n = x_n)$ where $(x_1, x_2, \ldots, x_n)$ is a binary $n$-vector that has exactly $y$ ONEs in it. For example, with $n = 4$, the event $Y=3$ is comprised of the union of $\binom{4}{3} = 4$ different events

$$(X_1 = 0, X_2 = 1, X_3 = 1, X_4 = 1) \implies (x_1, x_2, x_3, x_4) = 0111,\\ (X_1 = 1, X_2 = 0, X_3 = 1, X_4 = 1) \implies (x_1, x_2, x_3, x_4) = 1011,\\ (X_1 = 1, X_2 = 1, X_3 = 0, X_4 = 1) \implies (x_1, x_2, x_3, x_4) = 1101,\\ (X_1 = 1, X_2 = 1, X_3 = 1, X_4 = 0) \implies (x_1, x_2, x_3, x_4) = 1110.$$ Furthermore, each of these events has probability $p^3(1-p)$, or more generally, $p^y(1-p)^{n-y}$. In short, $$P(Y = y) = \underbrace{p^y(1-p)^{n-y} + p^y(1-p)^{n-y} + \cdots + p^y(1-p)^{n-y}}_{\binom ny ~ \text{terms}} = \binom{n}{y}p^y(1-p)^{n-y}.$$

Turning to your formula $$P(X_1 = x_1, ... , X_n = x_n \mid Y = y) = \frac{p^y(1-p)^{n-y}}{\binom{n}{y} p^y(1-p)^{n-y}} =\frac{1}{\binom{n}{y}} \text{ if } \sum_{i=1}^{n}x_i = y,$$ note that it would be better expressed as \begin{align} P(X_1 = x_1, ... , X_n = x_n \mid Y = y) &= \frac{P(X_1 = x_1, ... , X_n = x_n, Y = y)}{P(Y=y)}\\ &= \begin{cases}\displaystyle\frac{p^y(1-p)^{n-y}}{\binom{n}{y} p^y(1-p)^{n-y}} = \frac{1}{\binom{n}{y}}& \text{ if } \sum_{i=1}^{n}x_i = y,\\ \\ \displaystyle\frac{0}{\binom{n}{y}} = 0 & \text{ if } \sum_{i=1}^{n}x_i \neq y. \end{cases} \end{align} If you wish to read a deep meaning into this simple application of the definition on conditional probability to the binomial random variable $Y$ and its constituent independent identically distributed Bernoulli random variables, feel free to do so.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.