I would like to know if the MLE is still consistent, asymptotically normal, and efficient when I put restrictions on the parameter space.

I think my confusion stems from the definition of the parameter space. I know that the theorems that give us these results depend on certain properties of the parameter space. But does the parameter space mean the "natural values" it can take or the parameter space I give it? For example, the parameter space for the true mean of a Bernouilli random variable is $[0,1]$ (this is what I refer to as the "natural values") but if, for some reason, the true mean can only take values over the interval $(.25, .75)$ what is my parameter space now? If I find the MLE over the interval $(.25,.75)$ do I get the nice asymptotic MLE properties?


The nice properties stop working if the true value is on the boundary of your parameter space --- that, and certain regularity conditions on the likelihood itself. I believe that all you need is for the true value of the parameter to be within an open set of the parameter space. In your example, if the true value of $p$ is 0.10, then it's impossible with respect to your restricted parameter space, so of course everything will fail. But if it's an interior point of (.25,.75), then the mle will still be the usual $\hat{p}$ and the nice asymptotic properties will hold. And if $p=0.25$, you won't get the nice asymptotics either.

This is not a purely academic question. In mixed effects models, we often want to test if the random effect variance is 0, but under the null hypothesis that it is 0, the usual mle asymptotics no longer apply.

  • $\begingroup$ Does this just come down to the extreme value theorem? Like, either it's a mode or it's on the boundary. Or is it more subtle than that? $\endgroup$ – shadowtalker Mar 14 '15 at 2:34
  • $\begingroup$ Thanks for the response! "I believe that all you need is for the true value of the parameter to be within an open set of the parameter space." is basically what I would to know for sure. Also, to clarify my example, the true value of the mean has to be between .25 and .75. I edited my question. $\endgroup$ – TrynnaDoStat Mar 14 '15 at 3:24
  • $\begingroup$ @ssdecontrol the theorem at issue draws on the extreme value theorem, but the proof itself goes further. You need to show that the maximum of the likelihood function converges to the true parameter as the sample size increases. The math involved goes beyond first year calculus. $\endgroup$ – Placidia Mar 14 '15 at 22:03
  • $\begingroup$ @Placidia ok thanks. But is the intuition at least the same? $\endgroup$ – shadowtalker Mar 14 '15 at 22:04
  • $\begingroup$ Not really. The intuition is closer to that of a Taylor Series expansion. The proof of the CLT uses the characteristic function of the distribution. Since the obs are independent, the CF of the mean is basically the product of the CF's (scaled), and the log likelihood is a sum. You do a second order expansion of this thing, make some limiting arguments and reach the CF of the Normal. QED. $\endgroup$ – Placidia Mar 20 '17 at 12:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.