# Finding the form $g(T(\mathbf{y}), \lambda) \times h(\mathbf{y})$ for sufficiency statistic examples

I'm studying some notes that present examples of sufficiency:

Let $$Y_1, \dots, Y_n$$ be i.i.d. $$N(\mu, \sigma^2)$$. Note that $$\sum_{i = 1}^n (y_i - \mu)^2 = \sum_{i = 1}^n (y_i - \bar{y})^2 + n(\bar{y} - \mu)^2$$. Hence

\begin{align} L(\mu, \sigma; \mathbf{y}) &= \prod_{i = 1}^n \dfrac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{1}{2\sigma^2}(y_i - \mu)^2} \\ &= \dfrac{1}{(2\pi \sigma^2)^{n/2}}e^{-\frac{1}{2\sigma^2}\sum_{i = 1}^n (y_i - \bar{y})^2}e^{-\frac{1}{2\sigma^2}n(\bar{y} - \mu)^2} \end{align}

From Theorem 1, it follows that where $$T(\mathbf{Y}) = (\bar{Y}, \sum_{i = 1}^n (Y_i - \bar{Y})^2)$$ is a sufficient statistic for $$(\mu, \sigma)$$.

Theorem 1 is presented as follows:

A statistic $$T(\mathbf{Y})$$ is sufficient for $$\theta$$ if, and only if, for all $$\theta \in \theta$$

$$L(\theta; \mathbf{y}) = g(T(\mathbf{y}), \theta) \times h(\mathbf{y})$$

where the function $$g(\cdot)$$ depends on $$\theta$$ and the statistic $$T(\mathbf{Y})$$, while the function $$h(\cdot)$$ does not contain $$\theta$$.

Theorem 1 implies that if the likelihood $$L(\theta; \mathbf{y})$$ depends on the data only through $$T(\mathbf{y})$$, $$T(\mathbf{Y})$$ is a sufficient statistic for $$\theta$$ and $$h(\mathbf{y}) \equiv 1$$.

For reference to another example, here is a Poisson example that I recently posted:

Let $$Y_1, \dots, Y_n$$ be a i.i.d. $$\text{Pois}(\lambda)$$. Then

\begin{align} L(\lambda; \mathbf{y} &= \prod_{i = 1}^n e^{-\lambda} \dfrac{\lambda^{y_i}}{y_i!} \\ &= e^{-\lambda n} \dfrac{\lambda^{\sum_{i = 1}^n y_i}}{\prod_{i = 1}^n y_i!} \\ &= g(T(\mathbf{y}), \lambda) \times h(\mathbf{y}) \end{align}

where $$T(\mathbf{y}) = \sum_{i = 1}^n y_i$$, $$g(T(\mathbf{y}), \lambda) = e^{-\lambda n} \lambda^{T(\mathbf{y})}$$ and $$h(\mathbf{y}) = \dfrac{1}{\prod_{i = 1}^n y_i!}$$

There are three things that I don't understand here:

1. How is it that $$\sum_{i = 1}^n (y_i - \mu)^2 = \sum_{i = 1}^n (y_i - \bar{y})^2 + n(\bar{y} - \mu)^2$$? EDIT: Answered here.

2. If, for $$L(\theta; \mathbf{y})$$, we require the form $$g(T(\mathbf{y}), \theta) \times h(\mathbf{y})$$, then, for $$L(\mu, \sigma; \mathbf{y})$$, what form do we require? Trying to think of this myself, I thought of three potentially correct forms: $$g(T(\mathbf{y}), (\mu, \sigma)) \times h(\mathbf{y})$$, $$g(T(\mathbf{y}), (\sigma, \mu)) \times h(\mathbf{y})$$, or $$g(T(\mathbf{y}), \mu, \sigma) \times h(\mathbf{y})$$.

3. Related to 2., comparing the first example to the Poisson example, I don't understand the conclusion of the first example. How does $$T(\mathbf{Y}) = (\bar{Y}, \sum_{i = 1}^n (Y_i - \bar{Y})^2)$$ satisfy the form $$g(T(\mathbf{y}), \lambda) \times h(\mathbf{y})$$?

I would greatly appreciate it if people would please take the time to clarify these points.

1. You can check immediately that the vectors $$\begin{bmatrix} y_1 - \bar{y}\\ \vdots \\ y_n - \bar{y}\\ \end{bmatrix}, \begin{bmatrix} \bar{y} -\mu\\ \vdots \\ \bar{y} -\mu\\ \end{bmatrix}$$ are orthogonal in $$\mathbb{R}^n$$.

2. Follow what the theorem says: $$\theta = (\mu, \sigma)$$. So $$g(T(\mathbf{y}), \theta) \times h(\mathbf{y})$$ becomes $$g(T(\mathbf{y}), (\mu, \sigma)) \times h(\mathbf{y}).$$

3. Take $$h(\mathbf{y}) = 1$$.

The geometric meaning of your example is the following. The density of the multivariate normal distribution $$\mathcal{N}( \mu \cdot \begin{bmatrix} 1\\ \vdots \\ 1\\ \end{bmatrix}, \sigma^2 I)$$ is constant on concentric spheres centered at $$\begin{bmatrix} \mu\\ \vdots \\ \mu\\ \end{bmatrix}$$. Now take an affine hyperplane $$V$$ in $$\mathbb{R}^n$$ that is orthogonal to line $$L$$ parametrized by $$\mu \cdot \begin{bmatrix} 1\\ \vdots \\ 1\\ \end{bmatrix}, \;\; \mu \in \mathbb{R}.$$ Suppose $$V$$ intersects $$L$$ at the point $$\begin{bmatrix} a\\ \vdots \\ a\\ \end{bmatrix}$$, then $$y \in V$$ if and only if its sample mean $$\bar{y} = a$$. Therefore the statistic $$T(y) = (\bar{y} = a, \sum_{i = 1}^n (y_i - \bar{y})^2)$$ corresponds to an $$n-1$$-dimensional sphere lying in $$V$$ of fixed radius $$\sqrt{\sum_{i = 1}^n (y_i - a)^2}$$ centered at $$\begin{bmatrix} a\\ \vdots \\ a\\ \end{bmatrix}. \;\;$$

It is clear from the geometry that, independent of $$(\mu, \sigma^2)$$, the density of any distribution in your family is constant on such $$n-1$$-dimensional spheres---therefore $$h(\mathbf{y}) = 1$$. This means that condition on the statistic $$T(y)$$, data is uniformly distributed independent of the parameter.

I'll try to make the answers a little bit more approachable. For your first question, you might take a brute-force approach: \begin{aligned} \sum (y_i - \mu)^2 &= \sum (y_i^2 - 2\mu y_i + \mu^2) \\ &=\sum (y_i^2 - 2\mu y_i + \mu^2 + \bar y^2 - \bar y^2 + 2\bar y y_i - 2\bar y y_i) \\ &= \sum\Big[(y_i - \bar y)^2 + \mu^2 - 2\mu y_i - \bar y^2 + 2\bar y y_i\Big] \\ &= \sum(y_i - \bar y)^2 + n\mu^2 - 2n\mu \bar y - n\bar y^2 + 2n \bar y^2 \\ &= \sum(y_i - \bar y)^2 + n(\mu^2 - 2\mu \bar y + \bar y^2) \\ &= \sum(y_i - \bar y)^2 + n(\mu - \bar y)^2 \end{aligned} It's simple algebra.
To answer your second question: the order of the elements in the parameter vector are unimportant, i.e., we might treat $$\theta$$ as $$[\mu, \sigma]$$ or $$[\sigma, \mu]$$; it doesn't matter. So all three forms are right.
Lastly, for your third question, "take $$h(y) = 1$$" means that $$g(T(y), \theta) = \frac{1}{(2\pi \sigma^2)^{n/2}}e^{-\frac{1}{2\sigma^2}\sum_{i = 1}^n (y_i - \bar{y})^2}e^{-\frac{1}{2\sigma^2}n(\bar{y} - \mu)^2}.$$
Notice that in the expression of $$g(T(y), \theta)$$, the only statistics that appear are $$\bar y$$ and $$\sum_{i=1}^n (y_i - \bar y)$$. The data interact, so to say, with the parameters only through these statistics. They are jointly sufficient.
• Even if the two vectors $(\mu, \sigma)$ and $(\sigma, \mu)$ are equally valid, are you sure that $g(T(\mathbf{y}), \mu, \sigma) \times h(\mathbf{y})$ is the same as the other two $g(T(\mathbf{y}), (\mu, \sigma)) \times h(\mathbf{y})$ and $g(T(\mathbf{y}), (\sigma, \mu)) \times h(\mathbf{y})$? The first one has the two variables $\sigma$ and $\mu$ as elements of the function $g$, whereas the other two have the vectors $(\mu, \sigma)$ or $(\sigma, \mu)$ as an element of the function $g$. Apr 13, 2021 at 18:19