5
$\begingroup$

According to the PDF here: https://www.math.arizona.edu/~tgk/466/sufficient.pdf, the sum of a sample of data is not a sufficient statistic for the normal distribution when the variance is unknown. Instead, the sum and sum of squares are jointly sufficient statistics. Now, let's say we have a game with two players. Both of them know that five samples are drawn from some normal distribution. None of them know the parameters of the normal distribution used to generate the data. The goal of the game is to estimate the mean of the normal distribution. The player that comes closer to the true mean wins 1\$ (absolute difference between estimated value and actual value is the objective function).

While the first player is given all five samples, the second one is given just the sum of the samples (and he knows there were five of them).

If the sum alone is not a sufficient statistic when the variance is unknown, then what strategy can the first player apply that uses all five of the data points to win money over multiple such games?

As a follow-up, let's say the distribution isn't normal anymore and the players know what it is (without knowing the parameters). And they still have to estimate the real mean from the information they're given. Are there examples of distributions where the first player would have a substantial advantage?

And finally, what if both players had no idea what distribution was used to generate the samples? Now would the first player have any advantage?

$\endgroup$
0

2 Answers 2

14
$\begingroup$

$\bar X$ is not a sufficient statistic because it does not contain all the information about $(\mu,\sigma^2)$, which is what it would mean for it to be sufficient.

However, $\bar X$ does contain all the information about $\mu$ in the sample, whether or not $\sigma^2$ is known. For example, $\bar X$ attains the Cramèr-Rao bound. Similarly, if $\mu$ is not known, $s^2$ contains all the information about $\sigma^2$ (though not if $\mu$ is known since $(\mu-\bar X)^2$ has information about $\sigma^2$). Having all the information about parts of a parameter is a more complicated property than sufficiency, though it has been studied (see e.g., Sprott 1975).

$\endgroup$
3
  • $\begingroup$ Thanks. What about for non normal distributions? $\endgroup$
    – ryu576
    Commented Jul 18, 2021 at 13:05
  • $\begingroup$ @ryu576 That sounds like another question. If this post answers your question, you should accept it and then write a new question about the non-normal case. $\endgroup$
    – Galen
    Commented Jul 18, 2021 at 16:22
  • 1
    $\begingroup$ I did ask that in this question. But sure, can ask another one if you think that's better. $\endgroup$
    – ryu576
    Commented Jul 18, 2021 at 16:50
2
$\begingroup$

There is a relationship between the sample range and the sample standard deviation. It is not as good as using the sufficient statistic, but not useless.

[The somewhat bogus 'rule' of thumb, mentioned in some elementary tests, that $S$ is well-estimated by the range divided by 5 or 6, is not what I have in mind; for normal data, the appropriate divisor depends crucially on sample size.]

Simulation in R: For a sample of size $n = 10$ from a normal distribution, the sample range divided by $k = 3.164$ is approximately equal to the sample standard deviation.

set.seed(2021)
m = 10^6;  n=10;  rng = s = numeric(m)
for(i in 1:m) {
 x = rnorm(n, 100, 10)  # mean irrelevant
 rng[i] = diff(range(x));  s[i] = sd(x)
 }
k = rng/s;  k
[1] 3.164182

Smaller number of points for plot:

Range = rng[1:10000];  StDev = s[1:10000]
plot(StDev, Range, pch=".")
 abline(a=0, b=k, col="green2", lwd=2)

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.