# Sample mean is always an optimal estimator of the mean?

Suppose we have $$T_i,i=1..n$$ i.i.d. with unknown distribution and we want to estimate $$E[T]$$. Note that in this setting we are not estimating E[T] as a parameter of a parameter-dependent family of distributions, therefore it is difficult to attach a meaning to a likelyhood function:

$$L[E[T]]=P(T_i|E[T])$$

as it would be done when for example estimating $$\mu$$ from maximum likelyhood, knowing that the underlying distribution is Gaussian.

Again what we would probably do is computing $$\overline{T}=\frac{1}{n}\sum_i T_i$$. And we would be sure that the estimator would be consistent and its variance would go to zero. For example we have trivially:

$$E[\overline{T}]=E[T_i]$$

and:

$$\sigma^2[\overline{T}]=O(1/n)$$

supposing that $$T_i$$ has finite variance.

Here is the question: can we prove that $$\overline{T}$$ is optimal in some way? To me conceptes from MLE estimators or sufficient statistics are a bit difficult to apply, since $$E[T]$$ is not a parameter of the distribution but maybe I am missing something? Can we "derive" the sample mean estimator to be optimal according to some criterion in the general case and without making assumptions on the underlying distributions ?

• Hi: This concept-issue is discussed in any good math stat class or book so I can't answer here but the article at the link below might help. Essentially, the MLE has attractive properties regardless of the underlying distribution. ocw.mit.edu/courses/mathematics/… – mlofton Apr 12 '19 at 18:01
• Thx a lot for the link I will read it carefully. But are you sure that it addresses my question? It seems to start from the assumption that we are trying to estimate a parameter from a distribution, like the first example I proposed for the mean of a gaussian distribution. But my question is about estimating the expectation value of the distribution, i.e. $E[X]$ without assumptions on the underlying distribution. How are the two things connected? To me they are a bit different? – Thomas Apr 12 '19 at 18:35
• Hi: I don't think it's possible to do what you're asking without an assumption about the underlying distribution. the mean is only "optimal" when the density has a certain form but maybe someone else can understand your question more deeply and have some insight. I also don't understand the line $L(E(T) = P(T_{i} | E(T))$ so maybe I'm totally not following. – mlofton Apr 13 '19 at 15:02
• Note that expectation of the distribution still involvsd the density so I'm pretty sure that the answer is no. – mlofton Apr 13 '19 at 15:03
• Thanks for your answer.The line indicates what, when estimating the mean $\mu$ from a gaussian distribution, would be called the likeleyhood function $L(\mu)$ en.wikipedia.org/wiki/Likelihood_function , i.e. the probability of the data observed, given the value of $\mu$. Yes the point is that these theories do not apply straightforwardly and I was looking for some other "theoretical" derivation of the sample mean estimator in the general case. – Thomas Apr 13 '19 at 15:07

What you stated in your original post is that the sample mean converges to the true mean in probability (in fact, it does with probability one) if expectations and variances are finite, i.e., the Law of Large Numbers. Together with unbiasedness, I think this makes the sample mean "optimal enough". No distributional assumptions are needed except finite first and second moments. Notice that this works equally well to estimate probabilities of events, since $$P(E) = E[I_E]$$. Yes, if you want to claim that the sample mean is the MLE then you need distributional assumptions.