Determine probability of an event using maximum likelihood estimation

Question

Problem

I have a bag of many red and green balls. To find out the ratio between the two, I randomly picked balls with replacement. Out of the 100 outcomes, 60 were red balls and 40 were green balls. Say the red ball ratio is $x$, and the green ball ratio is $1-x$. I want to use the Maximum Likelihood Estimation to determine $x$.

---

I tried the following: The probability of picking a red ball is $x$. $$ X_b= \begin{cases} 60, & \text{if}\ b=R \\ 40, & \text{otherwise} \end{cases} $$ An estimator of maximum likelihood is given by $$L(X_i|x) = x^{X_i} (1-x)^{1-X_i}$$ If we pick a red ball from our sample, we will have $L(X_R|x) = x^{X_R} (1-x)^{1-X_R}$

\begin{align*} \frac{\partial}{\partial x} L(X_R|x) = 0 \implies \frac{\partial}{\partial x} \log L(X_R|x) &= \frac{X_R}{x} - \frac{1-X_R}{1-x} = 0 \end{align*}

Thus, $x = X_R = 60$

I don't know where I am mistaken. I believe that the ratio of red balls should be $x = 0.6$ and that of green balls should be $1-x = 0.4$. Because the probability of picking red ball is $P(X_R) = \frac{|R|}{|\Omega|} = \frac{60}{100} = 0.6$

Answer 1

The problem is that you are not writing the likelihood in the right way. Let's say that $X_i=1$ if the ball $i$ is red, and $X_i=0$ if it is green. Then, the probability for a single trial is

$$P(X_i|x) = x^{X_i} (1-x)^{1-X_i}$$

Now, you perform $N$ observations. The joint probability of making all those observations is (and here comes the big difference with your approach)

$$P( \{X_i \}|x) = \prod_i^Nx^{X_i} (1-x)^{1-X_i}$$

With respect to the parameter $x$, this is the likelihood function you want to maximize. For simplicity, we can maximize its logarithm which simplifies the math and does not change the final result (since the logarithm is a monotonically growing function). Thus,

$$L(x) = \log(P(\{X_i \}|x)) = \sum_i^N \left[ X_i\log x + (1-X_i)\log (1-x) \right]$$

Now, we can split the sum based on the color of the balls, and take into account that you observed $n_R$ red balls and $n_G$ green balls. We also use that $X_i=0$ for the green balls and that $1-X_i=0$ for the red ones. We obtain

$$L(x) = \log(P(\{X_i \}|x)) = n_R \log x + n_G \log (1-x) $$

Differentiating with respect to $x$ and equalling to zero, we obtain the maximum likelihood estimation $\hat{x}$ for $x$,

$$ \hat{x} = \frac{n_R}{n_G + n_R}$$

which was to be expected.