0
$\begingroup$

From the connotation of "Maximum likelihood estimator" I am inclined to think that the maximum likelihood estimator of the mean of a distribution should equal the mean of the sample values drawn from that distribution. What else could the "maximum likelihood estimate" of the mean be?

Also, by calculus, the least squares estimator of the mean is again equal to the mean of the sample values of a sample drawn from the distribution. So is the MLE of the mean of a distribution always equal to the least squares estimator of the mean? If it is not, can someone give a counter-example?

$\endgroup$
  • 4
    $\begingroup$ The lognormal distribution family provides a familiar counterexample. $\endgroup$ – whuber Nov 12 '19 at 15:37
  • $\begingroup$ i can imagine fat tailed distributions (pareto?) where sample mean is definitely not a good estimate of the mean. But my calculus is pretty much nonexistent, so I don't know if that is the correct answer. $\endgroup$ – rep_ho Nov 12 '19 at 16:37
  • 5
    $\begingroup$ Consider Laplace distribution, whose MLE is sample median. $\endgroup$ – Zhanxiong Nov 12 '19 at 17:27
  • $\begingroup$ @whuber Why did you not post the following as an answer? "The lognormal distribution family provides a familiar counterexample." $\endgroup$ – user17144 Nov 13 '19 at 8:10
  • 1
    $\begingroup$ Because Xi'an has already supplied that answer along with a clear explanation. $\endgroup$ – whuber Nov 13 '19 at 13:26
5
$\begingroup$

A generic contradiction to your intuition is that the MLE is invariant by transformations, while the mean is not. In particular, in exponential families, the MLE is the empirical mean of the natural statistics, but not of other transforms of the sample. For instance, in a Normal $X\sim \mathcal N(\theta,1)$ sample, the MLE of $\theta$, mean of $X$, is $X$, but the MLE of the mean of $\exp(X)$, $\exp\{\theta+1/2\}$, is $\exp\{X+1/2\}$ and not $\exp\{X\}$.

See also the connected discussion on when is the MLE a biased estimator of the mean.

$\endgroup$
  • $\begingroup$ Can you please see my comments addressed to whuber right below the question? I think the reason for the MLE not being the sample average for lognormal is that the algebra for making them equal is messed up, and not that the mean is not invariant under transformations. Even if the mean were invariant under transformation, the MLE for the lognormal mean would still be only the GM of the sample and not the AM..In any case, thank you for your answer. $\endgroup$ – user17144 Nov 14 '19 at 12:32
0
$\begingroup$

maximum likelihood estimator of the mean of a distribution

I don't think I've ever seen the mean be computed via MLE. Remember, MLE is about parameters, not moments of the distribution.

For a lot of distributions, the parameters just happen to be best estimated by the sample mean (see $\mu$ for the normal, $\lambda$ for the poisson), but this isn't always the case (see $\lambda$ for the exponential, but this depends on the parameterization).

$\endgroup$
  • 4
    $\begingroup$ The mean can always be considered a parameter. What might illuminate your point would be an example of a family of distributions in which all have the same mean. A Normal$(0,\sigma^2)$ family would work fine for that purpose, for instance. $\endgroup$ – whuber Nov 12 '19 at 15:44
  • 1
    $\begingroup$ How will this example illuminate Demetri's point? $\endgroup$ – user17144 Nov 12 '19 at 15:51
  • 2
    $\begingroup$ In this example, the MLE of the mean is $0,$ which will almost surely not equal the sample mean. $\endgroup$ – whuber Nov 12 '19 at 16:18
  • 2
    $\begingroup$ @Cagdas Many authors conceive of a "parameter" as a (reasonably nicely behaved) function $\theta:\Omega\to\mathbb{R}$ where $\Omega$ is the set of possible probability distributions (such as the Normal ones). In this sense not only are $\mu$ and $\sigma^2$ parameters of the Normal$(\mu,\sigma^2)$ family, but so are $\sigma$ (the SD), $1/\sigma^2$ (the precision), the skewness, and so on. Typically a parameter is estimated directly, but in MLE it suffices to estimate any set of numbers that identify one distribution $\hat F\in\Omega$ uniquely, for then $\hat\theta=\theta(\hat F).$ $\endgroup$ – whuber Nov 12 '19 at 20:28
  • 2
    $\begingroup$ @whuber I understand. I have learned something new today. Thank you. $\endgroup$ – Cagdas Ozgenc Nov 12 '19 at 20:48
0
$\begingroup$

Following @ZhanXiong's Comment. Suppose we look at $n = 10^5$ samples of size $n = 5$ from a Laplace (double exponential) population centered at $10.$ That is, population mean and median are both 10.

The following simulation in R, illustrates that the sample means $\bar X = A$ and $\tilde X = H$ have $E(A) = E(H) = 10,$ so that both the sample mean and median are unbiased estimators of the center. However, the sample means have a larger standard deviation than the sample medians. Thus, according to one frequently-used criterion, the sample median is a "better" estimator of the center than the sample mean.

set.seed(1112)
m = 10^5;  n = 5
x = rexp(m*n)-rexp(m*n)+10
DTA = matrix(x, nrow=m)
a = rowMeans(DTA)
mean(a);  sd(a)
[1] 9.997945          # aprx E(A) = 10 
[1] 0.6317852         # aprx SD(A) = sqrt(2/5) =  0.6325

h = apply(DTA,1,median)
mean(h);  sd(h)
[1] 9.997512          # aprx E(H) = 10
[1] 0.5910876         # SD(H) < SD(A)

enter image description here

par(mfrow=c(2,1))
 hist(a, prob=T, br=40, col="skyblue2", xlim=c(6,15), 
      main="Aprx Dist'n of Sample Meane")
 hist(h, prob=T, br=40, col="skyblue2", xlim=c(6,15), 
      main="Aprx Dist'n of Sample Medians")
par(mfrow=c(1,1))
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.