Understanding sufficient statistics geometrically

Question

Consider the distribution $\mathcal{P} = \mathcal{N}(\mu, 1)$, where the variance is known but the mean is unknown. Let $X_1,X_2\sim P$ i.i.d. In this case $T = X_1+X_2$ is a sufficient statistic.

I am trying to understand this geometrically. We know that $\begin{bmatrix}X_1\\X_2\end{bmatrix} \sim \mathcal{N} \left( \begin{bmatrix}\mu\\\mu\end{bmatrix}, \begin{bmatrix}1&0\\0&1\end{bmatrix} \right)$. This is a distribution centered somewhere on the line $x_2=x_1$ with circular contours.

Now if you were given $X_1+X_2=5$ say, you know that your observation lies somewhere on the line $x_1+x_2=5$. However I don't see how the probability of a specific observation, $P(X_1=3, X_2=2|T=5; \mu)$ does not depend on $\mu$. Clearly the probability is very different if $\begin{bmatrix}\mu\\\mu\end{bmatrix} = \begin{bmatrix}3\\3\end{bmatrix}$ vs. if $\begin{bmatrix}\mu\\\mu\end{bmatrix}=\begin{bmatrix}100\\100\end{bmatrix}$.

I understand the binomial distribution case (see example 2 in http://www.stat.cmu.edu/~larry/=stat705/Lecture5.pdf) but I don't see a similar intuition for the normal distribution. What am I missing?

Answer 1

Although the likelihood of the outcome is indeed different for different values of $\mu$, the sufficiency condition means that all outcomes on the line $x_1+x_2=5$ have the same likelihood as each other, for any given value of $\mu$. So the constancy that is at issue in the sufficiency condition is the fact that the points on the line do not differ from each other in inferential terms --- this does not mean that variation of the parameter does not affect the likelihood.

Geometrically, you can visualise this by looking at a 3D surface plot of the likelihood (or equivalently, the log-likelihood), as a function of $x_1$ and $x_2$. The condition of sufficiency means that the lines of values for fixed values of the sufficient statistic correspond to "ridges" on the surface plot. So, for example, when $T=5$ you are looking at the line of values where $x_1+x_2=5$, and this line gives a ridge of constant values for the likelihood function in the surface plot. If you vary $\mu$ then this will change the likelihood function in the surface plot, but the lines of interest will still be ridges will constant values on the surface plot.

Answer 2

There's a little trick to it. Let's just work with the squared term in the exponent of $\exp\{-{1 \over 2}\sum(x_i-\mu)^2\}$, as everything else is a constant.

$$\begin{eqnarray} \sum_{i=1}^2(x_i-\mu)^2 &=& (x_1 - \bar{x} + \bar{x} - \mu)^2 + (x_2 - \bar{x} + \bar{x} - \mu)^2 \\ &=& \sum_i(x_i-\bar{x})^2 + 2(\bar{x}-\mu)^2 +2(x_1-\bar{x})(\bar{x}-\mu)+2(x_2-\bar{x})(\bar{x}-\mu)\\ &=& \sum_i(x_i-\bar{x})^2 + 2(\bar{x}-\mu)^2 +2(x_1 + x_2-2\bar{x})(\bar{x}-\mu) \end{eqnarray}$$ Since $\bar{x} = (x_1+x_2)/2$, $x_1+x_2-2\bar{x} = 0$. This zeroes out the last term on the r.h.s., leaving us with:

$$\sum_i(x_i-\mu)^2 = \sum_i(x_i-\bar{x})^2 + 2(\bar{x}-\mu)^2$$

Since the last term on the r.h.s. is a constant given $x_1+x_2$, it gets absorbed into the constant of integration, leaving us with:

$$p(x_1,x_2|T,\mu) \propto e^{-{1 \over 2}\sum_i(x_i-T/2)^2}$$

which is evidently independent of $\mu$; the constant of integration that remains to be calculated is what is required to make the above expression integrate to $1$, so clearly it too must be independent of $\mu$.