Finding MLE of $p$ where $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$

Question

Let $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$ be independent Bernoulli random variables where $p\in[0,1/3]$. Derive the MLE of $p$.

We have that

$$L(p\mid \vec{x})=p^{x_1}(1-p)^{1-x_1}(3p)^{x_2}(1-3p)^{1-x_2}$$

Upon taking the natural log of both sides we get

$$\mathscr{L}(p\mid\vec{x})=x_1\text{log}(p)+(1-x_1)\text{log}(1-p)+x_2\text{log}(3p)+(1-x_2)\text{log}(1-3p)$$

Then

$$\begin{align*} \frac{\partial\mathscr{L}}{\partial p} &=\frac{x_1}{p}-\frac{1-x_1}{1-p}+\frac{x_2}{p}-\frac{3(1-x_2)}{1-3p}\\\\ &=\frac{6p^2-3x_1p-px_2-4p+x_1+x_2}{p(1-p)(1-3p)} \end{align*}$$

which equals zero when

$$p=\frac{3x_1+x_2+4\pm\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}$$

From here it's clear that

$$\frac{3x_1+x_2+4+\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}\geq\frac{8}{12}\gt\frac{1}{3}$$

and

$$0\leq \frac{3x_1+x_2+4-\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}\leq\frac{1}{3}$$

so

$$\hat{p}=\frac{3x_1+x_2+4-\sqrt{9x_1^2+6x_1x_2+x_2^2-16x_2+16}}{12}$$

However, a problem arises when trying to show that this is a global maximum

We have from software that

$$\begin{align*} \frac{\partial^2\mathscr{L}}{\partial p^2} &=\frac{-18p^4+24p^3+18x_1p^3+6p^3x_2-10p^2-21x_1p^2-13p^2x_2+8x_1p+8px_2-x_1-x_2}{p^2\left(-p+1\right)^2\left(-3p+1\right)^2} \end{align*}$$

Attempting to get a negative upper bound on the numerator to show that this is less than zero, we have that

$$-18p^4+24p^3+18x_1p^3+6p^3x_2-10p^2-21x_1p^2-13p^2x_2+8x_1p+8px_2-x_1-x_2$$

equals

$$-18p^4+p^3(24+18x_1+6x_2)+p^2(-10-21x_1-13x_2)+p(8x_1+8x_2)-x_1-x_2$$

which is less than or equal to

$$-18p^4+48p^3-10p^2+16p\leq\frac{48}{27}+\frac{16}{3}$$

so I fail to get a negative upper bound.

How can I show that what I obtained is a global maximum?

Answer 1

We have

$$\frac{\partial^2\mathscr{L}}{\partial p^2}=-\frac{x_1}{p^2}-\frac{1-x_1}{(1-p)^2}-\frac{x_2}{p^2}-\frac{9(1-x_2)}{(1-3p)^2}$$

which is clearly negative for any $p\in[0,1/3]$ so $\hat{p}$ is a global maximum.

Answer 2

Though your final answer is correct, I think the derivation of the MLE is much more simpler.

Given $x_1,x_2$, the likelihood function is

\begin{align} L(p\mid x_1,x_2)&=3^{x_2}p^{x_1+x_2}(1-p)^{1-x_1}(1-3p)^{1-x_2}\mathbf1_{x_1,x_2\in\{0,1\}} \\&=\begin{cases}3p(1-p)&,\text{ if }x_1=0,x_2=1 \\ p(1-3p)&,\text{ if }x_1=1,x_2=0 \\ (1-p)(1-3p)&,\text{ if }x_1=x_2=0 \\ 3p^2&,\text{ if }x_1=x_2=1 \qquad\qquad,\,0\le p\le\frac{1}{3} \end{cases} \end{align}

Study the cases separately:

• Observe that the curve $3p(1-p)$ is a parabola which is increasing in $p\in[0,1/3]$, so its maximum is reached at the boundary point $p=1/3$.

• Rewrite $p(1-3p)$ as $\frac{1}{12}-3(p-\frac{1}{6})^2$, so that it is maximized at $p=1/6$.

• It is obvious that $(1-p)(1-3p)$ is maximized for the minimum possible value of $p$, and $3p^2$ is maximized for the maximum possible value of $p$. So the maximization again occurs at the boundary points.

So maximizing $L(p\mid x_1,x_2)$ in each case subject to the constraint $0\le p\le 1/3$ yields the maximum likelihood estimator of $p$ :

\begin{align} \hat p(x_1,x_2)&=\begin{cases}\frac{1}{3}&,\text{ if }(x_1,x_2)=(0,1)\text{ or }(x_1,x_2)=(1,1) \\ \frac{1}{6}&,\text{ if }(x_1,x_2)=(1,0) \\ 0&,\text{ if }(x_1,x_2)=(0,0) \end{cases} \end{align}

This agrees with the functional form of $\hat p$ you had obtained. And I don't think it is required to express the MLE in a functional form. It is perfectly okay to give the answer as above.