What is the Likelihood function and MLE of Binomial distribution?

Question

I searched online, so many people mix up MLE of binomial and Bernoulli distribution.

They are saying: "If $X$ is $Binomial(N,\theta)$ then MLE is $\hat{\theta} = X/N$." And I don't agree with that, that is just MLE for Bernoulli in N trials, not for Binomial.

But I can't find correct Likelihood function and MLE of Binomial distribution online. So I try to derive it myself, and seek for confirmation here.

---

For Bernoulli, I know that::

its Likelihood function is $$\prod\limits_{i = 1}^n {{p_X}{{\left( {{x_i}} \right)}_{Ber\left( \theta \right)}}} = \left( {{\theta ^{\sum\limits_{i = 1}^n {{x_i}} }}{{\left( {1 - \theta } \right)}^{n - \sum\limits_{i = 1}^n {{x_i}} }}} \right)$$ its MLE is $${{\hat \theta }_{Ber\left( \theta \right)}} = \frac{{\sum\limits_{i = 1}^n {\left( {{x_i}} \right)} }}{n} = \bar x$$

${{p_X}\left( {{x_i}} \right)}$ is the pdf (or pmf)

---

For Binomial, I tried following::

its Likelihood function is

$$\left\{ \begin{array}{l}L\left( {\theta |{\bf{x}}} \right) = \prod\limits_{i = 1}^n {{p_X}{{\left( {{x_i}} \right)}_{Bin\left( {N,\theta } \right)}}} = \prod\limits_{i = 1}^n {\left( {\begin{array}{*{20}{c}}N\\{{x_i}}\end{array}} \right) \cdot {\theta ^{{x_i}}}{{\left( {1 - \theta } \right)}^{N - {x_i}}}} \\ = \prod\limits_{i = 1}^n {\left( {\begin{array}{*{20}{c}}N\\{{x_i}}\end{array}} \right)} \cdot \left( {\prod\limits_{i = 1}^n {{\theta ^{{x_i}}}{{\left( {1 - \theta } \right)}^{N - {x_i}}}} } \right)\\ = \prod\limits_{i = 1}^n {\left( {\begin{array}{*{20}{c}}N\\{{x_i}}\end{array}} \right)} \cdot \left( {{\theta ^{\sum\limits_{i = 1}^n {{x_i}} }}{{\left( {1 - \theta } \right)}^{\sum\limits_{i = 1}^n {\left( {N - {x_i}} \right)} }}} \right)\\ = \prod\limits_{i = 1}^n {\left( {\begin{array}{*{20}{c}}N\\{{x_i}}\end{array}} \right)} \cdot \left( {{\theta ^{\sum\limits_{i = 1}^n {{x_i}} }}{{\left( {1 - \theta } \right)}^{nN - \sum\limits_{i = 1}^n {{x_i}} }}} \right)\\\left[ {\left( {\begin{array}{*{20}{c}}N\\{{x_i}}\end{array}} \right){\text{ is just a constant when }}{x_i}{\text{ is given}}} \right.\\ \propto {\theta ^{\sum\limits_{i = 1}^n {{x_i}} }}{\left( {1 - \theta } \right)^{nN - \sum\limits_{i = 1}^n {{x_i}} }}\end{array} \right.$$

(I don't think we can have a general formula for the constant, so I drop it and just use the proportion, if you know please tell me.)

its MLE is $$\left\{ \begin{array}{l}\ln L = \ln \left( {\prod\limits_{i = 1}^n {\left( {\begin{array}{*{20}{c}}N\\{{x_i}}\end{array}} \right)} } \right) + \sum\limits_{i = 1}^n {\left( {{x_i}} \right)} \cdot \ln \left( \theta \right) + \left( {nN - \sum\limits_{i = 1}^n {{x_i}} } \right) \cdot \ln \left( {1 - \theta } \right)\\\frac{{d\left( {\ln L} \right)}}{{d\theta }} = 0 + \frac{{\sum\limits_{i = 1}^n {\left( {{x_i}} \right)} }}{\theta } - \frac{{nN - \sum\limits_{i = 1}^n {{x_i}} }}{{1 - \theta }}\\\frac{{d\left( {\ln L} \right)}}{{d\hat \theta }} = \frac{{\sum\limits_{i = 1}^n {\left( {{x_i}} \right)} }}{{\hat \theta }} - \frac{{nN - \sum\limits_{i = 1}^n {{x_i}} }}{{1 - \hat \theta }} = 0\\\left( {1 - \hat \theta } \right) \cdot \sum\limits_{i = 1}^n {\left( {{x_i}} \right)} = \left( {nN - \sum\limits_{i = 1}^n {{x_i}} } \right) \cdot \hat \theta \\\sum\limits_{i = 1}^n {\left( {{x_i}} \right)} = \left( {nN - \sum\limits_{i = 1}^n {{x_i}} + \sum\limits_{i = 1}^n {{x_i}} } \right) \cdot \hat \theta \\{{\hat \theta }_{Bin\left( {N,\theta } \right)}} = \frac{{\sum\limits_{i = 1}^n {\left( {{x_i}} \right)} }}{{nN}} = \frac{{\bar x}}{N}\end{array} \right.$$

Am I correct?

---

Also, n is the number of samples (trials), N is the number of times you flip it in 1 trial. ie::

in Bernoulli, you flip the coin n trials, you flip it 1 time each trial.

in Binomial, you flip the coin n trials, you flip it N times each trial.

(I guess this is why so many people mix these two up when calculating the Likelihood function)

Answer 1

These two models are statistically equivalent: $$ X_1,\dots,X_n \sim \text{Ber}(\theta), \quad \text{i.i.d.} $$ and $$ T \sim \text{Bin}(n, \theta). $$ the latter being the reduction of the former by sufficiency.

If you consider the following problem: $$ Y_1,\dots, Y_n \sim \text{Bin}(N,\theta), \quad \text{i.i.d.} $$ This is a different problem than either of the two above, a different model, not equivalent to the previous ones statistically. (This third model is equivalent to a version of the two above with $n$ replaced with $nN$.)

Saying "people mix up MLE of binomial and Bernoulli distribution." is itself a mix-up. There is no MLE of binomial distribution. Similarly, there is no MLE of a Bernoulli distribution. You have to specify a "model" first. Then, you can ask about the MLE. There many different models involving Bernoulli distributions. There are also many different models involving Binomial distributions. Once you fix a model, you can talk about the likelihood, etc.

Answer 2

Graphical comment: Suppose you have $X\sim\mathsf{Binom}(N, p),$ with $N=10$ and $x = 6$ successes.

p=seq(0,1, by=.01)
like = p^6*(1-p)^4
mle = p[like==max(like)];  mle
[1] 0.6

plot(p, like, type="l")
  abline(h=0, col="green2")
  abline(v = mle, col="red", lwd=2)

Suppose you have $X\sim\mathsf{Binom}(N, p),$ with $N=100$ and $x = 60$ successes.

p=seq(0,1, by=.01)
like = p^60*(1-p)^40
mle = p[like==max(like)];  mle
[1] 0.6
plot(p, like, type="l")
  abline(h=0, col="green2")
  abline(v = mle, col="red", lwd=2)

Notice that the curvature of the likelihood function at its maximum is much tighter than for $N = 10.$