# Maximum likelihood estimator for distribution with bound constraints

In class, we talked about finding Maximum Likelihood Estimators but there is something I don't think we talked about. How is the MLE different for distributions with constrained bounds?

For example, for an iid random sample $X_1,\ldots,X_n$, if we compare the MLE for $\theta$ from a Bernoulli distribution versus the MLE for $\theta$ from another Bernoulli distribution but with 0 < $\theta$ < $\frac 12$, how is the MLE affected?

I know the MLE for a Bernoulli($\theta$) with 0 < $\theta$ < 1 is $\frac{\Sigma x_i}{n}$.

I thought it was the same for the 0 < $\theta$ < $\frac 12$ case but thinking back to a Uniform distribution, the MLE will vary depending on the bounds. However, I don't think order statistics is helpful here. Does anyone have any ideas?

• Difference: If the parameter space is restricted compared with the natural parameter space, the maximum may occur on the boundary of the restricted space in which case it does not solve the likelihood equations, ie does not cancel the score function. Mar 12, 2018 at 20:28
• Part of the answer is trivial: you maximize the likelihood subject to the constraints. That's what MLE means. The non-trivial part often requires research: how do you assess the precision of estimates when the MLE lies on the boundary of the parameter space? Ordinarily this is done with likelihood ratios and a chi-squared approximation, but these approximations usually fail for constrained solutions. I would hope that answers not be limited to just the trivial aspect of the question.
– whuber
Mar 12, 2018 at 20:34
• stats.stackexchange.com/q/151655/119261 Jul 4, 2020 at 18:08

In general, but not always, what will happen is that the constrained MLE will be the closest possible value to the unconstrained MLE. To re-use your example, if $\sum x_i / n = 0.7$ but $0 < \theta \leq 0.5$, then the unconstrained MLE for $\theta$ is $0.7$ but the constrained MLE for $\theta$ is 0.5. This will always be the case if the log likelihood is concave in the parameter of interest.
Note that if you have a strict inequality constraint, for example, $0 < \theta < 0.5$, the constrained MLE doesn't exist, as there is no largest number less than $0.5$.
The following graph shows an example of this. The unconstrained MLE is at $6.4$, but, if we add a constraint at the red line ($5.5$), the new MLE is at $5.5$, despite the presence of a mode at $0.7$.
• How can the constrained MLE not exist but the unconstrained MLE exist? (Serious question, not rhetorical.) Also, if the constraints are real, e.g., the mean # of traffic accidents on the Bay Bridge per day $>0$, the MLE should be subject to the constraint; it's just making sure the parameter estimate lies within the region of possibility, so to speak. Well, that's a bad example, but I hope you see what I mean - the real MLE is the constrained one, not the unconstrained one. May 2, 2018 at 21:57