5
$\begingroup$

Suppose $X_1, X_2, \dots, X_n$ are iid Bernoulli(p) random variables. How do you find the restricted maximum likelihood for p where $0<p<0.5$?

My work so far:

Write out the likelihood: $L=p^{\sum x} (1-p)^{n-\sum x}$

Take the log: $\ln(L) = \sum x\ln(p)+(n-\sum x)\ln(1-p)$

Take the derivative and set equal to zero: $\frac{\sum x-np}{p(1-p)}=0$.

Now I'm stuck. I don't know how to incorporate the restrictions for $p$.

$\endgroup$
4
  • 1
    $\begingroup$ Please add the [self-study] tag & read its wiki. $\endgroup$ Commented May 10, 2015 at 15:01
  • 3
    $\begingroup$ When you minimize a function over a closed interval, like $[0, .5]$, the minimum is either in the interior, or it occurs at the endpoints. $\endgroup$ Commented May 10, 2015 at 16:21
  • $\begingroup$ I don't understand. The algebra still works out so $p=(\sum x)/n$, but that isn't any different from the unrestricted likelihood. I don't understand how the minimum will change by examining just the interval. $\endgroup$
    – user76754
    Commented May 10, 2015 at 16:36
  • $\begingroup$ Are there two cases, one where $\sum x/n>.5$ and one where $\sum x/n<.5$? If that's true, then in the first case the minimum will be when $p=0.5$, and in the second it will be when $p=\sum x/n$. Is that it? $\endgroup$
    – user76754
    Commented May 10, 2015 at 16:39

2 Answers 2

1
$\begingroup$

Continuing from our discussion in the comments:

When minimizing a function over a closed interval, like $[0, .5]$, the minimum is either in the interior, or it occurs at the endpoints.

As you say:

The algebra still works out so $p=\frac{\sum x}{n}$, but that isn't any different from the unrestricted likelihood. I don't understand how the minimum will change by examining just the interval.

True! So now you have two cases. It may be the case that your solution already meets the constraints (i.e. it lies in the interval $[0, .5]$), in which case, great! On the other hand, it may lie outside that interval. In this case, you must fall back on the "or it occurs at the endpoints" conditions. You now know that the optimum must be at an endpoint, so you have two points to check, and must simply choose the one that gives you the minimum likelihood.

Are there two cases, one where $p=\frac{\sum x}{n} > 0.5$ and one where ∑$p=\frac{\sum x}{n}$ < 0.5? If that's true, then in the first case the minimum will be when p=0.5, and in the second it will be when $p=\frac{\sum x}{n}$. Is that it?

As long as you're convinced that it is not possible for your function to be $\leq 0$, and that the point where the derivative vanishes is not actually a local maximum (I'm saying maximum to be consistent with our discussion, though I think it is possible you are looking to maximize your function, not minimize). To answer these questions, ask yourself: Does $p=0$ make sense in the context of your problem? What is the sign of the second derivative at the point where the derivative vanishes?

$\endgroup$
1
$\begingroup$

The likelihood function is (or is proportional to) $p^k(1-p)^{n-k}$ which has value $0$ at $p=0$ and at $p=1$. In between, as $p$ increases from $0$ towards $1$, the likelihood function rises monotonically to a peak at $p=\frac kn$ and then decreases monotonically to $0$. So, if you know that the value of $p$ is restricted to $[0,\frac 12]$, the question as to what value of $p$ maximizes $p^k(1-p)^{n-k}$ for $p\in [0,\frac 12]$ is easy to solve, and the answer is $$\hat{p}_{\scriptstyle{\text{MLE}}} = \min\left\{\frac kn, \frac 12\right\}.$$

$\endgroup$
1
  • 1
    $\begingroup$ What is $k$ here? $\endgroup$
    – Math1000
    Commented May 10, 2018 at 23:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.