Finding the form $g(T(\mathbf{y}), \lambda) \times h(\mathbf{y})$ for sufficiency statistic examples

Question

I'm studying some notes that present examples of sufficiency:

Let $Y_1, \dots, Y_n$ be i.i.d. $N(\mu, \sigma^2)$. Note that $\sum_{i = 1}^n (y_i - \mu)^2 = \sum_{i = 1}^n (y_i - \bar{y})^2 + n(\bar{y} - \mu)^2$. Hence

$$\begin{align} L(\mu, \sigma; \mathbf{y}) &= \prod_{i = 1}^n \dfrac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{1}{2\sigma^2}(y_i - \mu)^2} \\ &= \dfrac{1}{(2\pi \sigma^2)^{n/2}}e^{-\frac{1}{2\sigma^2}\sum_{i = 1}^n (y_i - \bar{y})^2}e^{-\frac{1}{2\sigma^2}n(\bar{y} - \mu)^2} \end{align}$$

From Theorem 1, it follows that where $T(\mathbf{Y}) = (\bar{Y}, \sum_{i = 1}^n (Y_i - \bar{Y})^2)$ is a sufficient statistic for $(\mu, \sigma)$.

Theorem 1 is presented as follows:

A statistic $T(\mathbf{Y})$ is sufficient for $\theta$ if, and only if, for all $\theta \in \theta$

$$L(\theta; \mathbf{y}) = g(T(\mathbf{y}), \theta) \times h(\mathbf{y})$$

where the function $g(\cdot)$ depends on $\theta$ and the statistic $T(\mathbf{Y})$, while the function $h(\cdot)$ does not contain $\theta$.

Theorem 1 implies that if the likelihood $L(\theta; \mathbf{y})$ depends on the data only through $T(\mathbf{y})$, $T(\mathbf{Y})$ is a sufficient statistic for $\theta$ and $h(\mathbf{y}) \equiv 1$.

For reference to another example, here is a Poisson example that I recently posted:

Let $Y_1, \dots, Y_n$ be a i.i.d. $\text{Pois}(\lambda)$. Then

$$\begin{align} L(\lambda; \mathbf{y} &= \prod_{i = 1}^n e^{-\lambda} \dfrac{\lambda^{y_i}}{y_i!} \\ &= e^{-\lambda n} \dfrac{\lambda^{\sum_{i = 1}^n y_i}}{\prod_{i = 1}^n y_i!} \\ &= g(T(\mathbf{y}), \lambda) \times h(\mathbf{y}) \end{align}$$

where $T(\mathbf{y}) = \sum_{i = 1}^n y_i$, $g(T(\mathbf{y}), \lambda) = e^{-\lambda n} \lambda^{T(\mathbf{y})}$ and $h(\mathbf{y}) = \dfrac{1}{\prod_{i = 1}^n y_i!}$

There are three things that I don't understand here:

1. How is it that $\sum_{i = 1}^n (y_i - \mu)^2 = \sum_{i = 1}^n (y_i - \bar{y})^2 + n(\bar{y} - \mu)^2$? EDIT: Answered here.

2. If, for $L(\theta; \mathbf{y})$, we require the form $g(T(\mathbf{y}), \theta) \times h(\mathbf{y})$, then, for $L(\mu, \sigma; \mathbf{y})$, what form do we require? Trying to think of this myself, I thought of three potentially correct forms: $g(T(\mathbf{y}), (\mu, \sigma)) \times h(\mathbf{y})$, $g(T(\mathbf{y}), (\sigma, \mu)) \times h(\mathbf{y})$, or $g(T(\mathbf{y}), \mu, \sigma) \times h(\mathbf{y})$.

3. Related to 2., comparing the first example to the Poisson example, I don't understand the conclusion of the first example. How does $T(\mathbf{Y}) = (\bar{Y}, \sum_{i = 1}^n (Y_i - \bar{Y})^2)$ satisfy the form $g(T(\mathbf{y}), \lambda) \times h(\mathbf{y})$?

I would greatly appreciate it if people would please take the time to clarify these points.

Answer 1

1. You can check immediately that the vectors $$ \begin{bmatrix} y_1 - \bar{y}\\ \vdots \\ y_n - \bar{y}\\ \end{bmatrix}, \begin{bmatrix} \bar{y} -\mu\\ \vdots \\ \bar{y} -\mu\\ \end{bmatrix} $$ are orthogonal in $\mathbb{R}^n$.

2. Follow what the theorem says: $\theta = (\mu, \sigma)$. So $$ g(T(\mathbf{y}), \theta) \times h(\mathbf{y}) $$ becomes $$ g(T(\mathbf{y}), (\mu, \sigma)) \times h(\mathbf{y}). $$

3. Take $h(\mathbf{y}) = 1$.

The geometric meaning of your example is the following. The density of the multivariate normal distribution $$ \mathcal{N}( \mu \cdot \begin{bmatrix} 1\\ \vdots \\ 1\\ \end{bmatrix}, \sigma^2 I) $$ is constant on concentric spheres centered at $\begin{bmatrix} \mu\\ \vdots \\ \mu\\ \end{bmatrix}$. Now take an affine hyperplane $V$ in $\mathbb{R}^n$ that is orthogonal to line $L$ parametrized by $$ \mu \cdot \begin{bmatrix} 1\\ \vdots \\ 1\\ \end{bmatrix}, \;\; \mu \in \mathbb{R}. $$ Suppose $V$ intersects $L$ at the point $ \begin{bmatrix} a\\ \vdots \\ a\\ \end{bmatrix} $, then $y \in V$ if and only if its sample mean $\bar{y} = a$. Therefore the statistic $$ T(y) = (\bar{y} = a, \sum_{i = 1}^n (y_i - \bar{y})^2) $$ corresponds to an $n-1$-dimensional sphere lying in $V$ of fixed radius $\sqrt{\sum_{i = 1}^n (y_i - a)^2}$ centered at $ \begin{bmatrix} a\\ \vdots \\ a\\ \end{bmatrix}. \;\; $

It is clear from the geometry that, independent of $(\mu, \sigma^2)$, the density of any distribution in your family is constant on such $n-1$-dimensional spheres---therefore $h(\mathbf{y}) = 1$. This means that condition on the statistic $T(y)$, data is uniformly distributed independent of the parameter.

Answer 2

I'll try to make the answers a little bit more approachable. For your first question, you might take a brute-force approach: $$\begin{aligned} \sum (y_i - \mu)^2 &= \sum (y_i^2 - 2\mu y_i + \mu^2) \\ &=\sum (y_i^2 - 2\mu y_i + \mu^2 + \bar y^2 - \bar y^2 + 2\bar y y_i - 2\bar y y_i) \\ &= \sum\Big[(y_i - \bar y)^2 + \mu^2 - 2\mu y_i - \bar y^2 + 2\bar y y_i\Big] \\ &= \sum(y_i - \bar y)^2 + n\mu^2 - 2n\mu \bar y - n\bar y^2 + 2n \bar y^2 \\ &= \sum(y_i - \bar y)^2 + n(\mu^2 - 2\mu \bar y + \bar y^2) \\ &= \sum(y_i - \bar y)^2 + n(\mu - \bar y)^2 \end{aligned}$$ It's simple algebra.

To answer your second question: the order of the elements in the parameter vector are unimportant, i.e., we might treat $\theta$ as $[\mu, \sigma]$ or $[\sigma, \mu]$; it doesn't matter. So all three forms are right.

Lastly, for your third question, "take $h(y) = 1$" means that $$ g(T(y), \theta) = \frac{1}{(2\pi \sigma^2)^{n/2}}e^{-\frac{1}{2\sigma^2}\sum_{i = 1}^n (y_i - \bar{y})^2}e^{-\frac{1}{2\sigma^2}n(\bar{y} - \mu)^2}.$$

Notice that in the expression of $g(T(y), \theta)$, the only statistics that appear are $\bar y$ and $\sum_{i=1}^n (y_i - \bar y)$. The data interact, so to say, with the parameters only through these statistics. They are jointly sufficient.