2
$\begingroup$

I have a log-likelihood function with two parameters: $\mathcal{l}(\theta_1,\theta_2)$.

When I run a simulation, the maximum likelihood estimator $\widehat{\theta}=[\widehat{\theta}_1,\widehat{\theta}_2]'$ yields a reasonable value for $\theta_1$ and an over-estimated value for $\theta_2$.

For example, if the true $\theta_1$ and $\theta_2$ are 10 and 15, respectively, then $\widehat{\theta}_1=9.97$ and $\widehat{\theta}_2=19.5$.

In my setup, the incorrect estimate is an expected event. That is, theoretically the estimator $\widehat{\theta}_2$ is inconsistent.

However, I want to know whether the MLE always produces an over-estimated value.

Hence, my question is that Is it possible to know whether the MLE over-estimates the parameters always?

$\endgroup$
5
  • $\begingroup$ From inconsistency you cannot infer that the estimate will always be too large for any finite sample. $\endgroup$ Aug 2, 2023 at 11:38
  • 1
    $\begingroup$ Along those lines, if you simulated 1000 MLEs from 1000 independent samples, all generated with known $\theta_i$ you'd have a good idea. $\endgroup$ Aug 2, 2023 at 12:10
  • $\begingroup$ The situation that the MLE overestimates 'always' is not very typical. Did you possibly mean the situation that the MLE is overestimating (or biased) on average? $\endgroup$ Aug 2, 2023 at 15:58
  • $\begingroup$ "In my setup, the incorrect estimate is an expected event. That is, theoretically the estimator $\hat{\theta}_2$ is inconsistent" This is unclear. Why is your estimator inconsistent? Is it a MLE (which is supposed to be consistent) or not? What do you mean by 'is an expected event'? $\endgroup$ Aug 2, 2023 at 16:01
  • $\begingroup$ @SextusEmpiricus Thank you for the answer. I made some subtle points in the question. As you said, what I meant is that the average of estimates from 1,000 simulation is bigger than the true value, and the `always' indicates that in many different data generating process, the average values are bigger than the true value. Also, in my setup, I know that the likelihood function is misspecified but the average value for $\widehat{\theta}_1$ seems unbiased. That is the exact situation. Thank you. $\endgroup$ Aug 3, 2023 at 3:47

1 Answer 1

2
$\begingroup$

``Is it possible to know whether the MLE over-estimates the parameters always?''

Yes, for specific distributions, this is possible to know.


An example is the MLE for a uniform distribution $$x_i \sim U(\theta,1)$$ which has as MLE the minimum of the sample which always overestimates the true parameter

$$\hat\theta = \min(x_1, x_2, \dots, x_n) > \theta $$

Another example is the MLE for the mean of a normal distribution, where we know that it is unbiased, and hence we know whether it overestimates (namely it doesn't overestimate).


Aside from those examples, more generally you will need to consider the distribution everytime again. I don't believe that there is an easy formula that anders you yes or no on the question whether the MLE always overestimates.

I imagine that it is possible that it is too difficult to compute/analyse the distributions, but one can always use some simulations and see whether the MLE is, for those simulations, always an overestimation of the true value.

I find it hard to come up with other situations than such cases as the uniform distribution whose domain is bounded and determined by parameters, so I guess that in practice, this method of simulations should handle most cases.

$\endgroup$
1
  • $\begingroup$ Thank you! Especially I forget that I am checking the average values of my simulation results. Your comments gave me some important points! $\endgroup$ Aug 3, 2023 at 3:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.