# MLE estimation: sensitive to data?

I am trying to estimate a MLE for an exponential distribution using fmincon in Maltab. I am having problem to estimate my parameter. For instance, I simulate an exponential distribution with a chosen parameter and then use the simulated data in my MLE. I therefore hope to find the same parameter that I generated my simulated data with. The problem that I have is that I can't get fmincon to evaluate the correct parameter...it always give me my upper bound (bound set prior to estimation) of my parameter. Why? I was told that a MLE can be very sensitive to the data...the structure of the data.

Any insights for me?

-
Why are you using fmincon - a numerical minimization procedure - in the first place? The MLE for the exponential distribution can be derived analytically (it is the sample mean mean). That being said, this question would be much easier to answer if you gave us the full code for the procedure that you're using (i.e. show us how you use fmincon). –  MånsT Aug 19 '12 at 13:01
My bet is that you are having numerical issues because you are working on the "raw" scale - for numerical ML you should always work on the log scale. Maximise the log density, not the density itself. Even though this has an analytic solution, you should still get close using numerical optimisation - else there is something suspect with the numerical optimisation (either the matlab routine or the way you've applied it). –  probabilityislogic Aug 20 '12 at 0:39

The density of the exponential model is

$$f(x) = \frac{1}{\theta} e^{-x/\theta}.$$

The log-likelihood $\ell$ of a sample of size $n$ is thus

$$\ell(\theta) = - n \log(\theta) -\frac{1}{\theta} \sum_{i=1}^n x_i.$$

The maximum likelihood estimate of the parameter $\theta$, say $\hat{\theta}$ is such that the derivative is 0. In other words

$$\frac{d\ell}{d\theta} \big\vert _{\hat{\theta}} = 0 = - \frac{n}{\hat{\theta}} + \frac{1}{\hat{\theta}^2}\sum_{i=1}^n x_i,$$

which solves out to $\hat{\theta} = \bar{x}$. In the case of the exponential distribution, the MLE is the sample mean. With this preamble you can:

1. Test whether the fmincon function works as you expect.
2. See that, just like the sample mean, MLEs cannot be exact estimators.
3. See that MLE is not intrinsically different from other estimators. Whatever you mean by "sensitive to the data", it is probably not a property of MLEs, but more of the particular model you try to fit.
-
There is a small slip-up in the algebra here. You've parametrized the exponential via a rate parameter, so the MLE is $1/\bar x$, which is consistent with your equations. –  cardinal Aug 19 '12 at 17:52
@cardinal thanks (+1), I'll fix that. –  gui11aume Aug 19 '12 at 22:02
just to add to point 3 - mle can be sensitive but only when the sample size is not much larger than the number of parameters –  probabilityislogic Aug 20 '12 at 1:02