5
$\begingroup$

Given a random variable $X\sim Bin(n,p)$, where $p$ is known $p\in (0,1)$ , $n$ is an unknown positive integer and $x\in\{0,1,2,....n\}$, what is the maximum likelihood estimator of $n$?

I found this answer on maths.stackexchange, but I cannot understand it. Why do we want the ratio to be less than 1? After the part that is mentioned that "it is less than 1" I am lost.

$\endgroup$
1
  • 2
    $\begingroup$ See here for the related problem where also $p$ is unknown. $\endgroup$ Apr 30, 2019 at 20:28

3 Answers 3

8
$\begingroup$

In this answer to the question $$\dfrac{{n+1 \choose x}p^x(1-p)^{n+1-x}}{{n \choose x}p^x(1-p)^{n-x}} = \dfrac{n+1}{n+1-x}(1-p)$$ represents the likelihood ratio $$\frac{L(n+1\mid x,p)}{L(n\mid x,p)}$$ If this ratio is larger than one (1), $${L(n+1\mid x,p)}>{L(n|\mid x,p)}$$ $-$ergo the likelihood increases$-$ and if it is smaller than one (1) $${L(n+1\mid x,p)}<{L(n\mid x,p)}$$ $-$ergo the likelihood decreases$-$. To find the maximum likelihood estimator of $n\in\Bbb N$, one need find the integer value of $n$ when the ratio crosses one, since $$\frac{L(n+1\mid x,p)}{L(n\mid x,p)}=\frac{1-p}{1-\dfrac{x}{n+1}}$$ is decreasing in $n\in\Bbb N$. To wit,

  • $n\mapsto x/(n+1)$ is decreasing,
  • $n\mapsto 1-x/(n+1)$ increasing,
  • $n\mapsto 1/\{1-x/(n+1)\}$ decreasing,

respectively. Hence, if $x/(n+1)>p$, i.e., if $n+1<x/p$, then ${L(n+1\mid x,p)}>{L(n\mid x,p)}$ while, if $x/(n+1)<p$, i.e., f $n+1>x/p$, then ${L(n+1|x,p)}<{L(n\mid x,p)}$. This means that $${L(\lfloor x/p\rfloor-1\mid x,p)}<{L(\lfloor,x/p\rfloor\mid x,p)}<{L(\lfloor x/p\rfloor+1\mid x,p)}$$

$\endgroup$
3
  • 1
    $\begingroup$ Why can't we say that the MLE of n is $ \dfrac{x}{p} -1 $ since it is the point that ratio crosses 1. But instead we choose $\bigg\lfloor\dfrac{x}{p}\bigg\rfloor$ as the MLE of n $\endgroup$
    – GAGA
    Apr 30, 2019 at 18:05
  • $\begingroup$ Thank you , one last question. if for example $p= \dfrac{1}{2}$ thus, the by solving $ \frac{L(n+1|x,p)}{L(n|x,p)} = 1 $ we would get $n = \dfrac{x}{p} -1 =\dfrac{x}{0.5} -1 = 2*x -1 $( which is integer for every value of x ) in this case MLE of n is $n = \dfrac{x}{0.5} -1 ? or n = \dfrac{x}{0.5} $ $\endgroup$
    – GAGA
    Apr 30, 2019 at 19:56
  • 1
    $\begingroup$ In this special case, both are MLE $\endgroup$
    – Xi'an
    Apr 30, 2019 at 20:13
4
$\begingroup$

As Xi'an correctly points out, this is a maximisation problem over integers, not real numbers. The objective function is quasi-concave, so we can obtain the maximising value by finding the point at which the (forward) likelihood ratio first drops below one. His answer shows you how to do this, and I have nothing to add to that excellent explanation. However, it is worth noting that discrete optimisation problems like this can also be solved by solving the corresponding optimisation problem in the reals, and then considering the discrete argument points around the real optima.


Alternative optimisation method: In this particular problem it is also possible to obtain the answer via consideration of the corresponding maximisation problem over the reals. To do this, suppose we generalise the binomial likelihood function to allow non-integer values of $n$, while preserving its quasi-concavity:

$$L_x(n) = \frac{\Gamma(n+1)}{\Gamma(n-x+1)} (1-p)^{n-x} \quad \quad \quad \text{for all real } n \geqslant x.$$

This objective function is a generalisation of the binomial likelihood function ---i.e., in the special case where $n \in \mathbb{N}$ it simplifies to the binomial likelihood function you are considering. The log-likelihood function is:

$$\ell_x(n) = \ln \Gamma (n+1) - \ln \Gamma (n-x+1) + (n-x) \ln (1-p).$$

The derivatives with respect to $n$ are:

$$\begin{equation} \begin{aligned} \frac{d \ell_x}{dn}(n) &= \psi (n+1) - \psi (n-x+1) + \ln (1-p) \\[10pt] &= \sum_{i=1}^x \frac{1}{n-x+i} + \ln (1-p), \\[10pt] \frac{d^2 \ell_x}{d n^2}(n) &= - \sum_{i=1}^x \frac{1}{(n-x+i)^2} < 0. \\[10pt] \end{aligned} \end{equation}$$

(The first derivative here uses the digamma function.) We can see from this result that the log-likelihood is a strictly concave function, which means the likelihood is strictly quasi-concave. The MLE for $n$ occurs at the unique critical point of the function, which gives an implicit function for the real MLE $\hat{n}$. It is possible to establish that $x/p-1 \leqslant \hat{n} \leqslant x/p$ (see below). This narrows down the discrete MLE to be the unique point in this interval if $x/p \notin \mathbb{N}$, or one of the two boundary points if $x/p \in \mathbb{N}$. This gives an alternative derivation of the maximising value in the discrete case.


Establishing the inequalities: We have already established that the score function (first derivative of the log-likelihood) is a decreasing function. The critical point occurs at the unique point where this function crosses the zero line. To establish the inequalities, it is therefore sufficient to show that the score is non-positive at the argument value $n = x/n-1$ and non-negative at the argument value $n = x/p$.

The first of these two inequalities is established as follows:

$$\begin{equation} \begin{aligned} \frac{d \ell_x}{dn}(x/p-1) &= \sum_{i=1}^x \frac{1}{x/p-1-x+i} + \ln (1-p) \\[10pt] &= \sum_{i=1}^x \frac{p}{x(1-p)+(i-1)p} + \ln (1-p) \\[10pt] &= \sum_{i=0}^{x-1} \frac{p}{x(1-p)+ip} + \ln (1-p) \\[10pt] &\geqslant \int \limits_0^x \frac{p}{x(1-p)+ip} \ di + \ln (1-p) \\[10pt] &= \Bigg[ \ln(x(1-p)+ip) \Bigg]_{i=0}^{i=x} + \ln (1-p) \\[10pt] &= \ln(x) - \ln(x(1-p)) + \ln (1-p) \\[10pt] &= \ln(x) - \ln(x) - \ln(1-p) + \ln (1-p) = 0. \\[10pt] \end{aligned} \end{equation}$$

The second inequality is established as follows:

$$\begin{equation} \begin{aligned} \frac{d \ell_x}{dn}(x/p) &= \sum_{i=1}^x \frac{1}{x/p -x+i} + \ln (1-p) \\[10pt] &= \sum_{i=1}^x \frac{p}{x(1-p)+ip} + \ln (1-p) \\[10pt] &\leqslant \int \limits_0^x \frac{p}{x(1-p)+ip} \ di + \ln (1-p) \\[10pt] &= \Bigg[ \ln(x(1-p)+ip) \Bigg]_{i=0}^{i=x} + \ln (1-p) \\[10pt] &= \ln(x) - \ln(x(1-p)) + \ln (1-p) \\[10pt] &= \ln(x) - \ln(x) - \ln(1-p) + \ln (1-p) = 0. \\[10pt] \end{aligned} \end{equation}$$

$\endgroup$
3
  • $\begingroup$ Hi @Ben, I'm currently studying your answer and I'd have a perhaps trivial question. I cannot work it the step from $\psi(n+1) - \psi(n-x+1)$ to $\sum_{i=1}^x \frac{1}{n-x-i}$. Would have pointers to where I can find materials to work it out? $\endgroup$
    – Pietro
    May 27, 2021 at 16:46
  • 1
    $\begingroup$ @Pietro: I think that is a fairly well-known "reflection" formula, but the derivation escapes me right now. I've just asked for a derivation in a question at Math.SE, so I'm sure they can hook us up. $\endgroup$
    – Ben
    May 27, 2021 at 22:49
  • $\begingroup$ Ben, thanks a lot for your answer! :) $\endgroup$
    – Pietro
    May 28, 2021 at 8:24
2
$\begingroup$

Binomial PMF is a discrete function of $n$, say $f(n)$, given others, i.e. $x,p$. We want to maximize it in terms of $n$. Typically, we would take the derivate and equate it to zero, but in discrete cases we shouldn't do that. This PDF is known to have its peak value(s) around its mean (not exactly but close). Its graph first increases, and then decreases. Sometimes, it stays constant for a little bit before decreasing. Therefore, the answer there considers the ratio $f(n+1)/f(n)$ and see if it is increasing or not. When the ratio is less than $1$, it means $f(n)$ is in the decreasing region, and the boundary $n$ is a candidate of ML estimate.

$\endgroup$
2
  • 1
    $\begingroup$ that boundary is of n is $ \dfrac{x}{p} -1 $ . But why do instead we choose $\bigg\lfloor\dfrac{x}{p}\bigg\rfloor$ as the MLE of n $\endgroup$
    – GAGA
    Apr 30, 2019 at 18:00
  • $\begingroup$ Because $n$ must be an integer. $\endgroup$ Aug 21, 2019 at 9:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.