3
$\begingroup$

Let $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$ be independent Bernoulli random variables where $p\in[0,1/3]$. Derive the MLE of $p$.

We have that

$$L(p\mid \vec{x})=p^{x_1}(1-p)^{1-x_1}(3p)^{x_2}(1-3p)^{1-x_2}$$

Upon taking the natural log of both sides we get

$$\mathscr{L}(p\mid\vec{x})=x_1\text{log}(p)+(1-x_1)\text{log}(1-p)+x_2\text{log}(3p)+(1-x_2)\text{log}(1-3p)$$

Then

$$\begin{align*} \frac{\partial\mathscr{L}}{\partial p} &=\frac{x_1}{p}-\frac{1-x_1}{1-p}+\frac{x_2}{p}-\frac{3(1-x_2)}{1-3p}\\\\ &=\frac{6p^2-3x_1p-px_2-4p+x_1+x_2}{p(1-p)(1-3p)} \end{align*}$$

which equals zero when

$$p=\frac{3x_1+x_2+4\pm\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}$$

From here it's clear that

$$\frac{3x_1+x_2+4+\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}\geq\frac{8}{12}\gt\frac{1}{3}$$

and

$$0\leq \frac{3x_1+x_2+4-\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}\leq\frac{1}{3}$$

so

$$\hat{p}=\frac{3x_1+x_2+4-\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}$$

However, a problem arises when trying to show that this is a global maximum

We have from software that

$$\begin{align*} \frac{\partial^2\mathscr{L}}{\partial p^2} &=\frac{-18p^4+24p^3+18x_1p^3+6p^3x_2-10p^2-21x_1p^2-13p^2x_2+8x_1p+8px_2-x_1-x_2}{p^2\left(-p+1\right)^2\left(-3p+1\right)^2} \end{align*}$$

Attempting to get a negative upper bound on the numerator to show that this is less than zero, we have that

$$-18p^4+24p^3+18x_1p^3+6p^3x_2-10p^2-21x_1p^2-13p^2x_2+8x_1p+8px_2-x_1-x_2$$

equals

$$-18p^4+p^3(24+18x_1+6x_2)+p^2(-10-21x_1-13x_2)+p(8x_1+8x_2)-x_1-x_2$$

which is less than or equal to

$$-18p^4+48p^3-10p^2+16p\leq\frac{48}{27}+\frac{16}{3}$$

so I fail to get a negative upper bound.

How can I show that what I obtained is a global maximum?

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ It's simpler if you just show that $\partial^2 l/\partial p^2 < 0$ everywhere, then of course it follows that it's negative at the point of interest too. No substitutions required! And that's simpler if you work with your first expression for $\partial l/\partial p = (x_1+x_2)/p - (1-x_1)/(1-p) -3(1-x_2)/(1-3p)$ rather than your second. Take the derivative; you'll get three terms, each of which is $\leq 0$ but not all three of which can equal $0$ at the same time. $\endgroup$ – jbowman Nov 9 '18 at 0:09
  • $\begingroup$ That's what I'm trying to do though, no? I'm trying to show that whatever $x_1,x_2,$ and $p$ are, we have that the second derivative is negative. $\endgroup$ – Remy Nov 9 '18 at 0:11
  • $\begingroup$ See the edit to my comment. $\endgroup$ – jbowman Nov 9 '18 at 0:12
  • $\begingroup$ Oh, okay that makes sense. I just found the second derivative by hand and it's clearly negative everywhere. I relied too much on software. Thanks! I will answer my own question. $\endgroup$ – Remy Nov 9 '18 at 0:19
1
$\begingroup$

We have

$$\frac{\partial^2\mathscr{L}}{\partial p^2}=-\frac{x_1}{p^2}-\frac{1-x_1}{(1-p)^2}-\frac{x_2}{p^2}-\frac{9(1-x_2)}{(1-3p)^2}$$

which is clearly negative for any $p\in[0,1/3]$ so $\hat{p}$ is a global maximum.

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ Accept it! You did get it right, after all! $\endgroup$ – jbowman Nov 9 '18 at 1:19
  • $\begingroup$ I will, it says I have to wait 2 days :) $\endgroup$ – Remy Nov 9 '18 at 1:22

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.