MLE estimation: sensitive to data?

Question

I am trying to estimate a MLE for an exponential distribution using fmincon in Maltab. I am having problem to estimate my parameter. For instance, I simulate an exponential distribution with a chosen parameter and then use the simulated data in my MLE. I therefore hope to find the same parameter that I generated my simulated data with. The problem that I have is that I can't get fmincon to evaluate the correct parameter...it always give me my upper bound (bound set prior to estimation) of my parameter. Why? I was told that a MLE can be very sensitive to the data...the structure of the data.

Any insights for me?

Answer 1

The density of the exponential model is

$$f(x) = \frac{1}{\theta} e^{-x/\theta}. $$

The log-likelihood $\ell$ of a sample of size $n$ is thus

$$ \ell(\theta) = - n \log(\theta) -\frac{1}{\theta} \sum_{i=1}^n x_i. $$

The maximum likelihood estimate of the parameter $\theta$, say $\hat{\theta}$ is such that the derivative is 0. In other words

$$ \frac{d\ell}{d\theta} \big\vert _{\hat{\theta}} = 0 = - \frac{n}{\hat{\theta}} + \frac{1}{\hat{\theta}^2}\sum_{i=1}^n x_i, $$

which solves out to $\hat{\theta} = \bar{x}$. In the case of the exponential distribution, the MLE is the sample mean. With this preamble you can:

1. Test whether the fmincon function works as you expect.
2. See that, just like the sample mean, MLEs cannot be exact estimators.
3. See that MLE is not intrinsically different from other estimators. Whatever you mean by "sensitive to the data", it is probably not a property of MLEs, but more of the particular model you try to fit.

Answer 2

If you expect the estimate from simulated data to be equal to the value of the parameter form the simulated distribution you are naive. That will never happen. This is not a question of sensitivity to data. It is the natural variability of estimates. The estimate have a distribution with a mean and a variance. So although I don't know the specifics of the program you are using, it would not be unusual to see the program provide a confidence interval along with the estimate to provide an idea of the uncertainty in the estimate.

Now maximum likelihood is based on a parametric form of the distribution ( the negative exponential in your case). So if the data appear to come from a distribution that is not exponential the estimate could be sensitive to the characteristics of the data set. As an illustrative example, the sample mean is the mle for the population mean of a normal distribution. But if the data contains a very large positive or negative observation the result will be very sensitive to that outlier.