A Question on Elementary Statistical Inference

Question

A box contains $5$ white and $2$ black balls. A coin with unknown $P(Head)=p$ is tossed once. If it lands HEADS then a white ball is added, else a black ball is added to the box. Then a ball is selected at random from the box. Given that the ball drawn is WHITE, find the Maximum Likelihood Estimator of $p$.

I find this problem quite confusing, really. It seems to be pretty straightforward and hence I am shocked by the substandard quality, else I am making some serious error. My attempt is as follows:

$P(White)=P(White|Head)P(Head)+P(White|Tail)P(Tail)=\dfrac{6}{8}.p+\dfrac{5}{8}(1-p)=\dfrac{p}{8}+\dfrac{5}{8}$

This is actually my likelihood of $p$ given the sample (my sample is WHITE ball). So this is maximized for $\hat{p}=1$. So $1$ (????) is the MLE for $p$. It is a constant estimator.

This is kind of weird. Any suggestion/correction/explanation is welcome.

Answer 1

Let $B$ denote an event of probability $p$. Then, the law of total probability says that $$\begin{align} P(A) &= P(A\mid B)P(B) + P(A\mid B^c)P(B^c)\\ &= P(A\mid B)\cdot p + P(A\mid B^c)\cdot (1-p) \end{align}$$ showing that $P(A)$ is a linear function of $p$, having value $P(A\mid B^c)$ when $p=0$ and value $P(A\mid B)$ when $p=1$. For $p\in (0,1)$, the value of $P(A)$ is somewhere between these extreme values. Thus, for $p \in [0,1]$, the maximum value of $P(A)$ is either $P(A\mid B)$ or $P(A\mid B^c)$ (except, of course, when $P(A\mid B) = P(A\mid B^c)$ -- which means that $A$ and $B$ are independent events -- and also means that $P(A)$ has the same value for all $p \in [0,1]$: knowledge that $A$ occurred is of no help in making inferences about the occurrence of $B$ or the value of $p$).

In this instance, $B$ is the event of tossing a Head on the coin and $A$ the event of drawing a White ball. Since $P(A\mid B) = \frac 68$ and $P(A\mid B^c) = \frac 58$ we have that $P(A)$ has maximum value $\frac 68$ when $p = 1$.

Answer 2

Your maximum likelihood estimate is correct. A more careful formalization may help your understanding of the concepts involved. In the following, take notice of how conditional independence is used.

Let $X=0,1$ be the result of the experiment, with $X=0$ meaning "Black ball is drawn", and $X=1$ meaning "White ball is drawn". Introduce a parameter $\Theta\in[0,1]$, and a random variable $Y$ such that $Y\mid\Theta=\theta\sim\mathrm{Bernoulli}(\theta)$, with $Y=1$ standing for "Heads".

The experiment can be formalized specifying the distribution of $X$ given $Y$ as $$ P(X=0\mid Y=0) = \frac{3}{8} \, , \qquad\qquad P(X=1\mid Y=0) = \frac{5}{8} \, , $$ $$ P(X=0\mid Y=1) = \frac{1}{4} \, ,\qquad\qquad P(X=0\mid Y=1) = \frac{3}{4} \, , $$ and postulating that $X$ and $\Theta$ are conditionally independent, given that $Y=y$.

Using the law of total probability and the product rule, the likelihood is $$ L_x(\theta) = P(X=x\mid \Theta=\theta) = \sum_{y=0,1} P(X=x, Y=y\mid \Theta=\theta) $$ $$ = \sum_{y=0,1} P(X=x\mid Y=y ,\Theta=\theta)\,P(Y=y\mid \Theta=\theta) $$ $$ = \sum_{y=0,1} P(X=x\mid Y=y)\,P(Y=y\mid \Theta=\theta) \, , $$ in which the last equality follows from the postulated conditional independence. Hence, the likelihood for your data is $$ L_1(\theta) = \frac{5}{8} \cdot (1-\theta) + \frac{3}{4} \cdot \theta = \frac{\theta}{8} + \frac{5}{8} \, , $$ and $\hat{\theta}_{\text{ML}}=1$.

The problem looks artificial because you are trying to estimate the parameter of the Bernoulli with just one observation.

Following Huber's suggestion, you may do a Bayesian analysis using the prior $\Theta\sim\mathrm{Beta}(a,b)$. The posterior distribution is a mixture of two betas. Can you find the Bayes estimate with quadratic loss?