3
$\begingroup$

I am currently studying sufficiency statistics. My notes say the following:

A statistic $T(\mathbf{Y})$ is sufficient for $\theta$ if, and only if, for all $\theta \in \Theta$, $$L(\theta; \mathbf{y}) = g(T(\mathbf{y}), \theta) \cdot h(\mathbf{y}),$$ where the function $g(\cdot)$ depends on $\theta$ and the statistic $T(\mathbf{Y})$, while the function $h(\cdot)$ does not contain $\theta$.

Sufficient statistics are not unique:
Any one-to-one transformation of a sufficient statistic is again a sufficient statistic.

Sufficiency depends on the model:

  • Let $Y_1, \dots, Y_n$ be a sample from $N(\mu, \sigma^2)$, where $\sigma^2 > 0$ is known. The only unknown parameter is $\mu = E[Y]$. $T(\mathbf{Y}) = \sum_{i = 1}^n Y_i$ or $T(\mathbf{Y}) = \bar{Y}$ are sufficient statistics for $\mu$.
  • Let $Y_1, \dots, Y_n$ be a sample from a $\text{Uniform}[ \mu - 1, \mu + 1]$ distribution. The only unknown parameter is $\mu = E[Y]$. In this case, $T(\mathbf{Y}) = \sum_{i = 1}^n Y_i$ or $T(\mathbf{Y}) = \bar{Y}$ are not sufficient statistics for $\mu$.

I don't understand why $T(\mathbf{Y}) = \sum_{i = 1}^n Y_i$ or $T(\mathbf{Y}) = \bar{Y}$ are sufficient statistics for $\mu$ for the normally distributed case but $T(\mathbf{Y}) = \sum_{i = 1}^n Y_i$ or $T(\mathbf{Y}) = \bar{Y}$ are not sufficient statistics for $\mu$ in the uniformly distributed case. I know that the unique characteristic of the uniform distribution is that its density is the same everywhere in the distribution, unlike the normal distribution, so I strongly suspect that this has something to do with it; although, as I said, I'm not sure precisely why.

An accompanying example for the uniformly distributed case is as follows:

Example
Let $Y_1, \dots, Y_n$ be an i.i.d. $U[\mu - 1, \mu + 1]$. It has the density $$f_\mu (y) = \begin{cases} 1/2 & \text{if} \ \mu - 1 \le y \le \mu + 1 \\ 0 & \text{otherwise}, \end{cases}$$ where $\mu \in \Theta = \mathbb{R} = (-\infty, \infty)$. The likelihood is given by $$\begin{align} L(\mu; \mathbf{y}) = \prod_{i = 1}^n f_{\mu} (y_i) &= \begin{cases} 1/2^n & \text{if} \ \mu - 1 \le y \le \mu + 1 \\ 0 & \text{otherwise} \end{cases} \\ &= \begin{cases} 1/2^n & \text{if} \ \mu - 1 \le y_{(1)} \le \dots \le y_{(n)} \le \mu + 1 \\ 0 & \text{otherwise} \end{cases} \\ &= \begin{cases} 1/2^n & \text{if} \ y_{(n)} - 1 \le \mu \le y_{(1)} + 1 \\ 0 & \text{otherwise} \end{cases} \end{align}$$ here $(y_{(1)} \le \dots \le y_{(n)})$ is the order statistic of $(y_1, \dots, y_n)$.

The only part of this example that is unclear to me is the last case:

$$\begin{cases} 1/2^n & \text{if} \ y_{(n)} - 1 \le \mu \le y_{(1)} + 1 \\ 0 & \text{otherwise} \end{cases}$$

Specifically, I don't understand where $\ y_{(n)} - 1 \le \mu \le y_{(1)} + 1$ came from; the equivalence of the algebra to the two cases that came before it are not clear to me.

I would greatly appreciate it if people would please take the time to clarify this.

$\endgroup$
5
  • 2
    $\begingroup$ It's saying that if all observations $y_i$ ( not $y$) are in $[\mu-1,\mu+1]$, then the minimum $y_{(1)}$ must be at least $\mu-1$ and the maximum $y_{(n)}$ must be at most $\mu+1$. Then rearranging to get the last line with $\mu$ in the middle. $\endgroup$ Mar 17, 2020 at 12:43
  • $\begingroup$ @StubbornAtom Thanks for the clarification. Any comments on my first point of confusion? $\endgroup$ Mar 17, 2020 at 12:59
  • 1
    $\begingroup$ Write down the joint density in each case and it will be clear from Factorization theorem. $\endgroup$ Mar 17, 2020 at 14:12
  • $\begingroup$ @StubbornAtom Can you please show how this is done? I'm not totally clear on this. $\endgroup$ Mar 22, 2020 at 0:06
  • $\begingroup$ It is not clear where exactly you are having trouble. These are standard examples which can be found in most textbooks. You can also find similar posts here and on the math site and discussions of sufficiency in general. So many similar questions have been answered before. $\endgroup$ Mar 23, 2020 at 7:37

1 Answer 1

3
+50
$\begingroup$

Normal distribution:

Let's compute the likelihood function in order to use the Factorization theorem

\begin{align} L(\mu; y) & = \prod_{i=1}^n \frac{1}{\sqrt{2\pi\sigma^2}} \exp \left (-\frac{(y_i-\mu)^2}{2\sigma^2} \right ) \\ &= (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left ( -\sum_{i=1}^n \frac{(y_i-\mu)^2}{2\sigma^2} \right ) \\ & = (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left (-\sum_{i=1}^n \frac{ \left ( \left (y_i-\overline{y} \right ) - \left (\mu-\overline{y} \right ) \right )^2}{2\sigma^2} \right ) \\ & = (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left( -{1\over2\sigma^2} \left(\sum_{i=1}^n(y_i-\overline{y})^2 + \sum_{i=1}^n(\mu-\overline{x})^2 -2\sum_{i=1}^n(y_i-\overline{y})(\mu-\overline{y})\right) \right) \\ &= (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left( -{1\over2\sigma^2} \left(\sum_{i=1}^n(y_i-\overline{y})^2 + \sum_{i=1}^n(\mu-\overline{x})^2 -2(\mu-\overline{y})\sum_{i=1}^n(y_i-\overline{y})\right) \right)\\ &= (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left( -{1\over2\sigma^2} \left (\sum_{i=1}^n(y_i-\overline{y})^2 + n(\mu-\overline{y})^2 \right ) \right ) \\ &= (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left( -{1\over2\sigma^2} \sum_{i=1}^n (y_i-\overline{y})^2 \right ) \exp \left (-\frac{n}{2\sigma^2} (\mu-\overline{y})^2 \right ) \end{align}

We can see that only the final term involves $\mu$ and $\bar{y}$ is the only statistics involved there. Hence for the normal distribution $\bar{y}$ is a sufficient statistics.

Uniform distribution:

However, for uniform distribution, as your note has illustrated,

$$L(\mu; y) = \frac1{2^n}\mathbb{1}_{\{y^{(n)}-1 \le \mu \le y^{(1)}+1\}}$$

We define $A_x = (x^{(n)}-1, x^{(1)}+1)$.

\begin{align} \frac{L(\mu; x)}{L(\mu; y)} = \begin{cases} 0 &, \mu \notin A_x, \mu \in A_y \\ 1 & ,\mu \in A_x, \mu \in A_y \\ \infty & ,\mu \in A_x, \mu \notin A_y \end{cases} \end{align}

This depends on $\mu$ unless $A_x = A_y$.

Hence $(y^{(1)}, y^{(n)})$ is a minimal sufficient.

Note that $(y^{(1)}, y^{(n)})$ is not determined by the average. Hence the sample average is not a sufficient statistics.

$\endgroup$
4
  • $\begingroup$ Can you please explain what's going on with $$L(\mu; y) = \frac1{2^n}\mathbb{1}_{\{y^{(n)}-1 \le \mu \le y^{(1)}+1\}}$$ and everything afterwards? I don't understand this part. $\endgroup$ Apr 15, 2021 at 14:44
  • $\begingroup$ I just write the form compactly using indicator variables. To be minimal sufficient, the ratio of $\frac{L(\mu; x)}{L(\mu; y)}$ should be independent from $\mu$. $\endgroup$ Apr 15, 2021 at 15:47
  • $\begingroup$ Oh, ok, I see. But how does $$\begin{align} \frac{L(\mu; x)}{L(\mu; y)} = \begin{cases} 0 &, \mu \notin A_x, \mu \in A_y \\ 1 & ,\mu \in A_x, \mu \in A_y \\ \infty & ,\mu \in A_x, \mu \notin A_y \end{cases} \end{align}$$ and its values $0$, $1$, $\infty$ relate to $\begin{cases} 1/2^n & \text{if} \ y_{(n)} - 1 \le \mu \le y_{(1)} + 1 \\ 0 & \text{otherwise} \end{cases}$? I understand how you got $0$, $1$, $\infty$ in your piecewise function, but I'm not clear on how it relates to my piecewise function. $\endgroup$ Apr 15, 2021 at 18:20
  • $\begingroup$ when $\mu \notin A_x, \mu \in A_y$, the numerator $L(\mu; x)$ would take value $0$, but the denominator, $L(\mu; y)$ would take value $\frac1{2^n}$. Your piecewise function is $L(\mu; x)$ and $L(\mu; y)$. $\endgroup$ Apr 16, 2021 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.