Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Consider a simple example of $X_{i}$ be i.i.d uniform distribution on the interval $[\theta,\theta+1]$. By strong low of large numbers, I may conclude that $$\overline{X}\rightarrow_{P} \theta+\frac{1}{2} $$ However, it is not so clear what is the pdf or cdf of $\overline{X}$ as I would have to do an $n$-dimensional integral to get the cdf as $F_{\overline{X}}(t)=P(\sum X_{i}<nt,\theta\le X_{i}<\theta+1)$. Nevertheless it seems "intuitive" that $\overline{X}$ should be a good statistic. But practice one soon learns that the real sufficient statistic is $(X_{(1)},X_{(n)})$, and proving this via the factorization theorem is not difficult.

I decided to ask the professor about this after the class. He told me that $\overline{X}$ is not a good statistic because it is not close enough to $\theta$, and as statisticans one has to consider real life applications. But this explanation is not persuasive, since I can use $\overline{X}-\frac{1}{2}$ for the same purpose. I want to ask if $\overline{X}$ is really a bad statistic for this example, and if yes for what reason. Also I want to know if $\overline{X}$ is a sufficient statistic.

share|improve this question
"the expectation" is not a statistic at all: $\bar X$ is a sample mean, not an expectation. Do you know the definition of a sufficient statistic, and any ways to check for sufficiency? You may find this discussion of the more general case of some help. (The short answer is '$\bar X$ is not sufficient'.) – Glen_b Feb 17 '14 at 4:37
@Glen_b: Yes, thanks for the reminder for correction. I do not know how to use it in $\overline{X}$ case as I did not see $\overline{X}$ appeared in the pdf $1_{X_{(1)}}1_{X_{(n)}}$. – Bombyx mori Feb 17 '14 at 4:42
I'm not sure what 'it' refers to in your second sentence. If you are referring to my link, the idea is to try to follow what's going on in the general case in order to understand in general terms how to apply the factorization theorem to uniform cases. – Glen_b Feb 17 '14 at 4:43
@Glen_b: I mean by the definition of a sufficient statistic or the factorization theorem. Thanks for the short answer - but how to show it? Should not SLLN imply it is a good statistic? – Bombyx mori Feb 17 '14 at 4:45
Here's something for you to ponder: consider random samples from a normal distribution where we average all of the data and where we average the first $\lceil{\sqrt n}\rceil$ observations. As $n\rightarrow\infty$, do both have almost sure convergence? – Glen_b Feb 17 '14 at 5:37
up vote 4 down vote accepted

Sufficiency pertains to data reduction, not estimation per se. This is an important distinction to understand. Yes, a "good" estimator is usually a function of a sufficient statistic, but that doesn't mean that all sufficient statistics are estimators.

As for your specific example, a simple way to understand why $\bar X$ is not a sufficient statistic for $\theta$ is to consider the following experiment: suppose I tell you $\bar X = 10$. Is this equivalent to all the information pertaining to $\theta$ that we can get from the sample? Of course not: for instance, $X_1 = 9.5, X_2 = 10.4, X_3 = 10.1$ could give us $10$, but so could $X_1 = X_2 = 9.75, X_3 = 10.5$. If you only have knowledge of $\bar X$, you have lost information about $\theta$ that was available in the original sample: namely, that in the first case, we must have $\theta \in [9.4,9.5]$, and in the second, $\theta \in [9.5,9.75]$. Notice those intervals are nearly disjoint. This is why $\bar X$ is insufficient for $\theta$. You may be able to use it to estimate $\theta$ when $n$ is large, but as I have pointed out, sufficiency has to do with data reduction, not estimation.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.