4
$\begingroup$

Consider the distribution $\mathcal{P} = \mathcal{N}(\mu, 1)$, where the variance is known but the mean is unknown. Let $X_1,X_2\sim P$ i.i.d. In this case $T = X_1+X_2$ is a sufficient statistic.

I am trying to understand this geometrically. We know that $\begin{bmatrix}X_1\\X_2\end{bmatrix} \sim \mathcal{N} \left( \begin{bmatrix}\mu\\\mu\end{bmatrix}, \begin{bmatrix}1&0\\0&1\end{bmatrix} \right)$. This is a distribution centered somewhere on the line $x_2=x_1$ with circular contours.

Now if you were given $X_1+X_2=5$ say, you know that your observation lies somewhere on the line $x_1+x_2=5$. However I don't see how the probability of a specific observation, $P(X_1=3, X_2=2|T=5; \mu)$ does not depend on $\mu$. Clearly the probability is very different if $\begin{bmatrix}\mu\\\mu\end{bmatrix} = \begin{bmatrix}3\\3\end{bmatrix}$ vs. if $\begin{bmatrix}\mu\\\mu\end{bmatrix}=\begin{bmatrix}100\\100\end{bmatrix}$.

I understand the binomial distribution case (see example 2 in http://www.stat.cmu.edu/~larry/=stat705/Lecture5.pdf) but I don't see a similar intuition for the normal distribution. What am I missing?

$\endgroup$

2 Answers 2

2
$\begingroup$

Although the likelihood of the outcome is indeed different for different values of $\mu$, the sufficiency condition means that all outcomes on the line $x_1+x_2=5$ have the same likelihood as each other, for any given value of $\mu$. So the constancy that is at issue in the sufficiency condition is the fact that the points on the line do not differ from each other in inferential terms --- this does not mean that variation of the parameter does not affect the likelihood.

Geometrically, you can visualise this by looking at a 3D surface plot of the likelihood (or equivalently, the log-likelihood), as a function of $x_1$ and $x_2$. The condition of sufficiency means that the lines of values for fixed values of the sufficient statistic correspond to "ridges" on the surface plot. So, for example, when $T=5$ you are looking at the line of values where $x_1+x_2=5$, and this line gives a ridge of constant values for the likelihood function in the surface plot. If you vary $\mu$ then this will change the likelihood function in the surface plot, but the lines of interest will still be ridges will constant values on the surface plot.

$\endgroup$
2
$\begingroup$

There's a little trick to it. Let's just work with the squared term in the exponent of $\exp\{-{1 \over 2}\sum(x_i-\mu)^2\}$, as everything else is a constant.

$$\begin{eqnarray} \sum_{i=1}^2(x_i-\mu)^2 &=& (x_1 - \bar{x} + \bar{x} - \mu)^2 + (x_2 - \bar{x} + \bar{x} - \mu)^2 \\ &=& \sum_i(x_i-\bar{x})^2 + 2(\bar{x}-\mu)^2 +2(x_1-\bar{x})(\bar{x}-\mu)+2(x_2-\bar{x})(\bar{x}-\mu)\\ &=& \sum_i(x_i-\bar{x})^2 + 2(\bar{x}-\mu)^2 +2(x_1 + x_2-2\bar{x})(\bar{x}-\mu) \end{eqnarray}$$ Since $\bar{x} = (x_1+x_2)/2$, $x_1+x_2-2\bar{x} = 0$. This zeroes out the last term on the r.h.s., leaving us with:

$$\sum_i(x_i-\mu)^2 = \sum_i(x_i-\bar{x})^2 + 2(\bar{x}-\mu)^2$$

Since the last term on the r.h.s. is a constant given $x_1+x_2$, it gets absorbed into the constant of integration, leaving us with:

$$p(x_1,x_2|T,\mu) \propto e^{-{1 \over 2}\sum_i(x_i-T/2)^2}$$

which is evidently independent of $\mu$; the constant of integration that remains to be calculated is what is required to make the above expression integrate to $1$, so clearly it too must be independent of $\mu$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.