2
$\begingroup$

I want to show that the statistic $\left(\sum_{i = 1}^n Y_i, \sum_{i = 1}^n Y_i^2 \right)$ is sufficient for $\mu$ but not minimal sufficient where $(Y_1, \dots, Y_n)$ is a random sample from $N(\mu, \mu)$ for $\mu > 0$.

I have already shown that the statistic $\sum_{i = 1}^n Y_i^2$ is minimal sufficient for $\mu$ where $(Y_1, \dots, Y_n)$ is a random sample from $N(\mu, \mu)$ for $\mu > 0$.

I began by calculating the joint density for $N(\mu, \sigma^2)$:

$$L(\mu, \sigma; \mathbf{y}) = \dfrac{1}{(2 \pi \sigma^2)^{n/2}} \exp{\left\{ -\dfrac{1}{2 \sigma^2} \sum_{i = 1}^n (y_i - \mu)^2 \right\}}$$

Switching to $N(\mu, \mu)$, I calculated the likelihood ratio to be

$$\dfrac{L(\mu; \mathbf{y_1})}{L(\mu; \mathbf{y_2})} = \exp{\left\{ \dfrac{1}{2 \mu} \left( \sum_{i = 1}^n y_{2i}^2 - \sum_{i = 1}^n y_{1i}^2 \right) + \left( \sum_{i = 1}^n y_{1i} - \sum_{i = 1}^n y_{2i} \right) \right\}}$$

This likelihood ratio does not depend on the parameter $\mu$ when $\sum_{i = 1}^n y_{2i}^2 = \sum_{i = 1}^n y_{1i}^2$:

$$\exp{\left\{ \dfrac{1}{2 \mu} \left( \sum_{i = 1}^n y_{2i}^2 - \sum_{i = 1}^n y_{1i}^2 \right) + \left( \sum_{i = 1}^n y_{1i} - \sum_{i = 1}^n y_{2i} \right) \right\}} = \exp{\left\{ \left( \sum_{i = 1}^n y_{1i} - \sum_{i = 1}^n y_{2i} \right) \right\}}$$

So, assuming I am understanding this correctly, if $\sum_{i = 1}^n Y_i^2$ was the only test statistic, then we could say that it is minimal sufficient.

Now, to account for $\sum_{i = 1}^n Y_i$, if we use $\sum_{i = 1}^n y_{2i} = \sum_{i = 1}^n y_{1i}$, then we get

$$\exp{\left\{ 0 \right\}} = 1$$

So, clearly, nothing changed between our calculations for the test statistic $\sum_{i = 1}^n Y_i^2$ and our calculation for the test statistic $\left(\sum_{i = 1}^n Y_i, \sum_{i = 1}^n Y_i^2 \right)$ – the likelihood ratio does not depend on the parameter $\mu$. What exactly are we supposed to do to show that $\left(\sum_{i = 1}^n Y_i, \sum_{i = 1}^n Y_i^2 \right)$ is sufficient but not minimal sufficient? What exactly is the argument that we're supposed to present?


I want to use the Fisher–Neyman factorization theorem, of the form $L(\mu; \mathbf{y}) = g(T(\mathbf{y}), \mu) \times h(\mathbf{y})$, to factor $\exp{\left\{ \dfrac{1}{2 \mu} \left( \sum_{i = 1}^n y_{2i}^2 - \sum_{i = 1}^n y_{1i}^2 \right) + \left( \sum_{i = 1}^n y_{1i} - \sum_{i = 1}^n y_{2i} \right) \right\}}$ and show that the statistic $\left(\sum_{i = 1}^n Y_i, \sum_{i = 1}^n Y_i^2 \right)$ is sufficient for $\mu$. So we immediately know that we have $T(\mathbf{Y}) = \left(\sum_{i = 1}^n Y_i, \sum_{i = 1}^n Y_i^2 \right)$. And since $\dfrac{1}{2 \mu} \left( \sum_{i = 1}^n y_{2i}^2 - \sum_{i = 1}^n y_{1i}^2 \right)$ has the parameter $\mu$, and $\left( \sum_{i = 1}^n y_{1i} - \sum_{i = 1}^n y_{2i} \right)$ has the data for $\sum_{i = 1}^n Y_i$, I would say that we require $g(T(\mathbf{y}), \mu) = \dfrac{1}{2 \mu} \left( \sum_{i = 1}^n y_{2i}^2 - \sum_{i = 1}^n y_{1i}^2 \right) + \left( \sum_{i = 1}^n y_{1i} - \sum_{i = 1}^n y_{2i} \right)$, and so we require that $h(\mathbf{y}) = 1$. This way, we get the Fisher-Neyman factorization

$$L(\mu; \mathbf{y}) = \left( \dfrac{1}{2 \mu} \left( \sum_{i = 1}^n y_{2i}^2 - \sum_{i = 1}^n y_{1i}^2 \right) + \left( \sum_{i = 1}^n y_{1i} - \sum_{i = 1}^n y_{2i} \right) \right) \times 1 = \left( \dfrac{1}{2 \mu} \left( \sum_{i = 1}^n y_{2i}^2 - \sum_{i = 1}^n y_{1i}^2 \right) + \left( \sum_{i = 1}^n y_{1i} - \sum_{i = 1}^n y_{2i} \right) \right)$$

And so this shows that $\left(\sum_{i = 1}^n Y_i, \sum_{i = 1}^n Y_i^2 \right)$ is a sufficient statistic.

$\endgroup$
11
  • $\begingroup$ Please may you clarify what it is you are conveying when you opt to represent the mean and variance parameter of the Gaussian $N(\mu, \sigma^2)$ with the single parameter $\mu$ in "$N(\mu, \mu)$"? Are you implying that they are equal? Or is it the case that one of the $\mu$ and $\sigma^2$ is known and the other known? In particular I am confused by the problem setup in the first line "I want to show...is sufficient for $\mu$ but not minimal sufficient...from $N(\mu, \mu)$ for $\mu > 0$. $\endgroup$
    – microhaus
    Apr 18, 2021 at 16:36
  • $\begingroup$ @microhaus I just used the $N(\mu, \sigma^2)$ random variables to more easily compute the joint distribution, since this is the conventional parameterisation. I then switched it to $N(\mu, \mu)$, which, I think, is just unknown mean and variance, but both mean and variance are equal to the unknown $\mu$ parameter. See this question stats.stackexchange.com/q/484819/163242 That user seems to be doing the same distribution, but their goal is different. In the beginning, they thought that the sufficient minimal statistic was $\{\sum x_i^2 , \sum x_i\}$, but then they realised [...] $\endgroup$ Apr 18, 2021 at 16:40
  • $\begingroup$ [...] that it was $\sum x_i^2$. The difference is that I'm trying to show that the statistic $\{\sum x_i^2 , \sum x_i\}$ ($\left(\sum_{i = 1}^n Y_i, \sum_{i = 1}^n Y_i^2 \right)$ in my notation) is sufficient but not minimal. So it seems that we're using the same distribution (that user calls theirs $N(\theta, \theta)$) but have different goals. $\endgroup$ Apr 18, 2021 at 16:45
  • 1
    $\begingroup$ The result is immediate from the definition of a minimal sufficient statistic: the second statistic is an (obvious) function of the latter but not vice versa. $\endgroup$
    – whuber
    Apr 18, 2021 at 16:51
  • 1
    $\begingroup$ @ThePointer. Yes all clear - I will put a formal definition in the answers of what whuber is saying from the notes I use. Perhaps that may clarify things. $\endgroup$
    – microhaus
    Apr 18, 2021 at 16:57

1 Answer 1

4
$\begingroup$

Proof strategy.

  1. Show that $T = \sum Y^2_i$ is minimal sufficient.

  2. Show that $U = (\sum Y_i, \sum Y^2_i)$ is sufficient.

  3. Use the following theorem to show that $T = g(U)$ for some function $g$.

  4. Show that you cannot find a function $h$ such that $U = h(T)$.

In order to supplement what @whuber is saying formally, here is a definition from Wasserman's All of Statistics:

Definition 9.3.5 A statistic $T$ is minimal sufficient if (i) it is sufficient, and (ii) it is a function of every other sufficient statistic.

In symbols, a statistic $T$ is minimal sufficient if it is the case that both:

  1. $T$ is sufficient.
  2. If $U$ is any other sufficient statistic, then $T = g(U)$ for some function $g$.

Once you have shown that $T$ is minimal sufficient and $U$ is sufficient, notice that if you define a scalar function $g : \mathbb{R}^2 \rightarrow \mathbb{R}$ that is the dot product between the row vector $(0, 1)$ and column vector $U \in \mathbb{R}^2$ like so:

$$g(U) = (0, 1) \cdot U$$

then $T = \sum_i Y^2_i = (0, 1) \cdot (\sum_i Y_i, \sum Y^2_i)^T = g(U)$.

In words, this function $g$ bins the redundant component (from the perspective of minimal sufficiency) of $U$ containing $\sum_i Y_i$.

How can we interpret this? You have already shown that $T$ is minimal sufficient in step 1. So this is not supplying any additional information on the minimal sufficiency of $T$.What this reveals however is an interpretation concerning minimal sufficiency as opposed to sufficiency.

Now $T$ and $U$ are similar in that both are sufficient. Informally, a sufficient statistic $S$ means that after computing $S = S(y_1,... y_n)$ to yield $S$, we can throw away the observations of the data set $Y_1 = y_1, \dots, Y_n = y_n$, and with the retention of the sufficient statistic $S$ alone, still be able to compute the likelihood function $L(\mu; y_1, \dots, y_n)$. As we have thrown the data away, we can no longer compute $L(\mu; y_1, \dots, y_n)$ directly, but can do so indirectly using $L(\mu; S)$.

So clearly, as both $T$ and $U$ are sufficient, we can use either to compute the likelihood function after having thrown away the data $y_1, \dots, y_n$, that is, we can compute $L(\mu; T)$ or $L(\mu; U)$. Imagine if we now ask, can we find a sufficient statistic that achieves the greatest possible reduction of the data and yet still allows us to compute the likelihood function? You can think of this as finding the most compressed summary of the data, with "minimal redudancy", in the sense that it contains just enough information to compute the likelihood function, and no more. This is an informal characterisation of minimal sufficiency.

Notice that the only difference between $T$ and $U$ is that $U$ contains an extra component containing $\sum_i Y_i$. From the perspective of minimal sufficiency, this additional component in $U$ is redundant, we can throw this component away and can still compute the likelihood function using $L(\mu; T)$. Hence $T$ achieves a more "compressed summary" of the data than $U$, but no so much that we cannot compute the likelihood function.

Now, is it possible to find a similar function $h$ such that $U = h(T)$? Notice if you give me $T = (\sum Y^2_i)$ and ask me to apply a function $h$ to this that gives me $U = (\sum Y_i, \sum Y_i^2)$, no such function $h$ exists. In particular, given only a statistic $T$ that is the sum of the squared observations, I cannot recover the sum of observations $\sum_i Y_i$. I therefore cannot recover the 1st component of $U$, and therefore I cannot recover $U$. Hence no function $h$ exists such that $U = h(T)$.

Now as we have established $U$ is sufficient, and because $T$ is also sufficient (being minimal sufficient), but we cannot find a function $h$ such that $U = h(T)$, we can conclude that $U$ is sufficient but not minimal sufficient.


Further comments.

  1. Your current proof of step 1. to show $T = \sum Y^2_i$ is minimal sufficient is missing a small formality. You also need to acknowledge the reverse implication (even if it's by inspection) that

$$T(Y_{11},..., Y_{1n}) = T(Y_{21}, ..., Y_{2n}) \implies L(\mu; \mathbf{y_1}) / L(\mu; \mathbf{y}_2) = c$$

  1. Use the factorisation theorem to show $U$ is sufficient.
$\endgroup$
6
  • $\begingroup$ Thanks for the answer! How do we go from 1. to 2.? That is, how do we show that $U = (\sum Y_i, \sum Y^2_i)$ is sufficient? $\endgroup$ Apr 18, 2021 at 17:15
  • 1
    $\begingroup$ @ThePointer. Have edited to address your comments, and to show explicitly what it means to find a function $g$ such that $T = g(U)$, and how this relates to binning redundant parts of the sufficient statistic to get the minimal sufficient statistic. $\endgroup$
    – microhaus
    Apr 18, 2021 at 17:51
  • $\begingroup$ Thanks again! I have edited my main post with my attempt at using the Fisher–Neyman factorization theorem. Is that correct? $\endgroup$ Apr 18, 2021 at 17:54
  • 1
    $\begingroup$ Small error - you need to apply factorisation theorem on $L(\mu; \mathbf{y})$. If I am correctly ascertaining that you've erroneously applied this to the ratio $L(\mu; \mathbf{y}_1) / L(\mu; \mathbf{y}_2)$, then this will need to be corrected. Note that the test for minimal sufficiency (via likelihood ratio) and sufficiency (via factorisation theorem) are distinct tests that shouldn't be confused. $\endgroup$
    – microhaus
    Apr 18, 2021 at 18:14
  • 1
    $\begingroup$ No sweat. It helps me too, as I am also studying this material. $\endgroup$
    – microhaus
    Apr 18, 2021 at 18:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.