5
$\begingroup$

Let $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$ be independent Bernoulli random variables where $p\in[0,1/3]$. Derive the MLE of $p$.

We have that

$$L(p\mid \vec{x})=p^{x_1}(1-p)^{1-x_1}(3p)^{x_2}(1-3p)^{1-x_2}$$

Upon taking the natural log of both sides we get

$$\mathscr{L}(p\mid\vec{x})=x_1\text{log}(p)+(1-x_1)\text{log}(1-p)+x_2\text{log}(3p)+(1-x_2)\text{log}(1-3p)$$

Then

$$\begin{align*} \frac{\partial\mathscr{L}}{\partial p} &=\frac{x_1}{p}-\frac{1-x_1}{1-p}+\frac{x_2}{p}-\frac{3(1-x_2)}{1-3p}\\\\ &=\frac{6p^2-3x_1p-px_2-4p+x_1+x_2}{p(1-p)(1-3p)} \end{align*}$$

which equals zero when

$$p=\frac{3x_1+x_2+4\pm\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}$$

From here it's clear that

$$\frac{3x_1+x_2+4+\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}\geq\frac{8}{12}\gt\frac{1}{3}$$

and

$$0\leq \frac{3x_1+x_2+4-\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}\leq\frac{1}{3}$$

so

$$\hat{p}=\frac{3x_1+x_2+4-\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}$$

However, a problem arises when trying to show that this is a global maximum

We have from software that

$$\begin{align*} \frac{\partial^2\mathscr{L}}{\partial p^2} &=\frac{-18p^4+24p^3+18x_1p^3+6p^3x_2-10p^2-21x_1p^2-13p^2x_2+8x_1p+8px_2-x_1-x_2}{p^2\left(-p+1\right)^2\left(-3p+1\right)^2} \end{align*}$$

Attempting to get a negative upper bound on the numerator to show that this is less than zero, we have that

$$-18p^4+24p^3+18x_1p^3+6p^3x_2-10p^2-21x_1p^2-13p^2x_2+8x_1p+8px_2-x_1-x_2$$

equals

$$-18p^4+p^3(24+18x_1+6x_2)+p^2(-10-21x_1-13x_2)+p(8x_1+8x_2)-x_1-x_2$$

which is less than or equal to

$$-18p^4+48p^3-10p^2+16p\leq\frac{48}{27}+\frac{16}{3}$$

so I fail to get a negative upper bound.

How can I show that what I obtained is a global maximum?

$\endgroup$
  • 1
    $\begingroup$ It's simpler if you just show that $\partial^2 l/\partial p^2 < 0$ everywhere, then of course it follows that it's negative at the point of interest too. No substitutions required! And that's simpler if you work with your first expression for $\partial l/\partial p = (x_1+x_2)/p - (1-x_1)/(1-p) -3(1-x_2)/(1-3p)$ rather than your second. Take the derivative; you'll get three terms, each of which is $\leq 0$ but not all three of which can equal $0$ at the same time. $\endgroup$ – jbowman Nov 9 '18 at 0:09
  • $\begingroup$ That's what I'm trying to do though, no? I'm trying to show that whatever $x_1,x_2,$ and $p$ are, we have that the second derivative is negative. $\endgroup$ – Remy Nov 9 '18 at 0:11
  • $\begingroup$ See the edit to my comment. $\endgroup$ – jbowman Nov 9 '18 at 0:12
  • $\begingroup$ Oh, okay that makes sense. I just found the second derivative by hand and it's clearly negative everywhere. I relied too much on software. Thanks! I will answer my own question. $\endgroup$ – Remy Nov 9 '18 at 0:19
2
$\begingroup$

We have

$$\frac{\partial^2\mathscr{L}}{\partial p^2}=-\frac{x_1}{p^2}-\frac{1-x_1}{(1-p)^2}-\frac{x_2}{p^2}-\frac{9(1-x_2)}{(1-3p)^2}$$

which is clearly negative for any $p\in[0,1/3]$ so $\hat{p}$ is a global maximum.

$\endgroup$
  • 1
    $\begingroup$ Accept it! You did get it right, after all! $\endgroup$ – jbowman Nov 9 '18 at 1:19
  • $\begingroup$ I will, it says I have to wait 2 days :) $\endgroup$ – Remy Nov 9 '18 at 1:22
1
$\begingroup$

Though your final answer is correct, I think the derivation of the MLE is much more simpler.

Given $x_1,x_2$, the likelihood function is

\begin{align} L(p\mid x_1,x_2)&=3^{x_2}p^{x_1+x_2}(1-p)^{1-x_1}(1-3p)^{1-x_2}\mathbf1_{x_1,x_2\in\{0,1\}} \\&=\begin{cases}3p(1-p)&,\text{ if }x_1=0,x_2=1 \\ p(1-3p)&,\text{ if }x_1=1,x_2=0 \\ (1-p)(1-3p)&,\text{ if }x_1=x_2=0 \\ 3p^2&,\text{ if }x_1=x_2=1 \qquad\qquad,\,0\le p\le\frac{1}{3} \end{cases} \end{align}

Study the cases separately:

  • Observe that the curve $3p(1-p)$ is a parabola which is increasing in $p\in[0,1/3]$, so its maximum is reached at the boundary point $p=1/3$.

  • Rewrite $p(1-3p)$ as $\frac{1}{12}-3(p-\frac{1}{6})^2$, so that it is maximized at $p=1/6$.

  • It is obvious that $(1-p)(1-3p)$ is maximized for the minimum possible value of $p$, and $3p^2$ is maximized for the maximum possible value of $p$. So the maximization again occurs at the boundary points.

So maximizing $L(p\mid x_1,x_2)$ in each case subject to the constraint $0\le p\le 1/3$ yields the maximum likelihood estimator of $p$ :

\begin{align} \hat p(x_1,x_2)&=\begin{cases}\frac{1}{3}&,\text{ if }(x_1,x_2)=(0,1)\text{ or }(x_1,x_2)=(1,1) \\ \frac{1}{6}&,\text{ if }(x_1,x_2)=(1,0) \\ 0&,\text{ if }(x_1,x_2)=(0,0) \end{cases} \end{align}

This agrees with the functional form of $\hat p$ you had obtained. And I don't think it is required to express the MLE in a functional form. It is perfectly okay to give the answer as above.

$\endgroup$

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.