# Finding MLE of $p$ where $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$

Let $$X_1\sim\text{Bernoulli}(p)$$ and $$X_2\sim\text{Bernoulli}(3p)$$ be independent Bernoulli random variables where $$p\in[0,1/3]$$. Derive the MLE of $$p$$.

We have that

$$L(p\mid \vec{x})=p^{x_1}(1-p)^{1-x_1}(3p)^{x_2}(1-3p)^{1-x_2}$$

Upon taking the natural log of both sides we get

$$\mathscr{L}(p\mid\vec{x})=x_1\text{log}(p)+(1-x_1)\text{log}(1-p)+x_2\text{log}(3p)+(1-x_2)\text{log}(1-3p)$$

Then

\begin{align*} \frac{\partial\mathscr{L}}{\partial p} &=\frac{x_1}{p}-\frac{1-x_1}{1-p}+\frac{x_2}{p}-\frac{3(1-x_2)}{1-3p}\\\\ &=\frac{6p^2-3x_1p-px_2-4p+x_1+x_2}{p(1-p)(1-3p)} \end{align*}

which equals zero when

$$p=\frac{3x_1+x_2+4\pm\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}$$

From here it's clear that

$$\frac{3x_1+x_2+4+\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}\geq\frac{8}{12}\gt\frac{1}{3}$$

and

$$0\leq \frac{3x_1+x_2+4-\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}\leq\frac{1}{3}$$

so

$$\hat{p}=\frac{3x_1+x_2+4-\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}$$

However, a problem arises when trying to show that this is a global maximum

We have from software that

\begin{align*} \frac{\partial^2\mathscr{L}}{\partial p^2} &=\frac{-18p^4+24p^3+18x_1p^3+6p^3x_2-10p^2-21x_1p^2-13p^2x_2+8x_1p+8px_2-x_1-x_2}{p^2\left(-p+1\right)^2\left(-3p+1\right)^2} \end{align*}

Attempting to get a negative upper bound on the numerator to show that this is less than zero, we have that

$$-18p^4+24p^3+18x_1p^3+6p^3x_2-10p^2-21x_1p^2-13p^2x_2+8x_1p+8px_2-x_1-x_2$$

equals

$$-18p^4+p^3(24+18x_1+6x_2)+p^2(-10-21x_1-13x_2)+p(8x_1+8x_2)-x_1-x_2$$

which is less than or equal to

$$-18p^4+48p^3-10p^2+16p\leq\frac{48}{27}+\frac{16}{3}$$

so I fail to get a negative upper bound.

How can I show that what I obtained is a global maximum?

• It's simpler if you just show that $\partial^2 l/\partial p^2 < 0$ everywhere, then of course it follows that it's negative at the point of interest too. No substitutions required! And that's simpler if you work with your first expression for $\partial l/\partial p = (x_1+x_2)/p - (1-x_1)/(1-p) -3(1-x_2)/(1-3p)$ rather than your second. Take the derivative; you'll get three terms, each of which is $\leq 0$ but not all three of which can equal $0$ at the same time. – jbowman Nov 9 '18 at 0:09
• That's what I'm trying to do though, no? I'm trying to show that whatever $x_1,x_2,$ and $p$ are, we have that the second derivative is negative. – Remy Nov 9 '18 at 0:11
• Oh, okay that makes sense. I just found the second derivative by hand and it's clearly negative everywhere. I relied too much on software. Thanks! I will answer my own question. – Remy Nov 9 '18 at 0:19

We have

$$\frac{\partial^2\mathscr{L}}{\partial p^2}=-\frac{x_1}{p^2}-\frac{1-x_1}{(1-p)^2}-\frac{x_2}{p^2}-\frac{9(1-x_2)}{(1-3p)^2}$$

which is clearly negative for any $$p\in[0,1/3]$$ so $$\hat{p}$$ is a global maximum.

• Accept it! You did get it right, after all! – jbowman Nov 9 '18 at 1:19
• I will, it says I have to wait 2 days :) – Remy Nov 9 '18 at 1:22

Though your final answer is correct, I think the derivation of the MLE is much more simpler.

Given $$x_1,x_2$$, the likelihood function is

\begin{align} L(p\mid x_1,x_2)&=3^{x_2}p^{x_1+x_2}(1-p)^{1-x_1}(1-3p)^{1-x_2}\mathbf1_{x_1,x_2\in\{0,1\}} \\&=\begin{cases}3p(1-p)&,\text{ if }x_1=0,x_2=1 \\ p(1-3p)&,\text{ if }x_1=1,x_2=0 \\ (1-p)(1-3p)&,\text{ if }x_1=x_2=0 \\ 3p^2&,\text{ if }x_1=x_2=1 \qquad\qquad,\,0\le p\le\frac{1}{3} \end{cases} \end{align}

Study the cases separately:

• Observe that the curve $$3p(1-p)$$ is a parabola which is increasing in $$p\in[0,1/3]$$, so its maximum is reached at the boundary point $$p=1/3$$.

• Rewrite $$p(1-3p)$$ as $$\frac{1}{12}-3(p-\frac{1}{6})^2$$, so that it is maximized at $$p=1/6$$.

• It is obvious that $$(1-p)(1-3p)$$ is maximized for the minimum possible value of $$p$$, and $$3p^2$$ is maximized for the maximum possible value of $$p$$. So the maximization again occurs at the boundary points.

So maximizing $$L(p\mid x_1,x_2)$$ in each case subject to the constraint $$0\le p\le 1/3$$ yields the maximum likelihood estimator of $$p$$ :

\begin{align} \hat p(x_1,x_2)&=\begin{cases}\frac{1}{3}&,\text{ if }(x_1,x_2)=(0,1)\text{ or }(x_1,x_2)=(1,1) \\ \frac{1}{6}&,\text{ if }(x_1,x_2)=(1,0) \\ 0&,\text{ if }(x_1,x_2)=(0,0) \end{cases} \end{align}

This agrees with the functional form of $$\hat p$$ you had obtained. And I don't think it is required to express the MLE in a functional form. It is perfectly okay to give the answer as above.