3
$\begingroup$

Given a random variable $X\sim Bin(n,p)$, where $p$ is known $p\in (0,1)$ , $n$ is an unknown positive integer and $x\in\{0,1,2,....n\}$, what is the maximum likelihood estimator of $n$?

I found this answer on maths.stackexchange, but I cannot understand it. Why do we want the ratio to be less than 1? After the part that is mentioned that "it is less than 1" I am lost.

$\endgroup$
  • 1
    $\begingroup$ See here for the related problem where also $p$ is unknown. $\endgroup$ – kjetil b halvorsen Apr 30 at 20:28
6
$\begingroup$

In this answer to the question $$\dfrac{{n+1 \choose x}p^x(1-p)^{n+1-x}}{{n \choose x}p^x(1-p)^{n-x}} = \dfrac{n+1}{n+1-x}(1-p)$$ represents the likelihood ratio $$\frac{L(n+1\mid x,p)}{L(n\mid x,p)}$$ If this ratio is larger than one (1), $${L(n+1\mid x,p)}>{L(n|\mid x,p)}$$ $-$ergo the likelihood increases$-$ and if it is smaller than one (1) $${L(n+1\mid x,p)}<{L(n\mid x,p)}$$ $-$ergo the likelihood decreases$-$. To find the maximum likelihood estimator of $n\in\Bbb N$, one need find the integer value of $n$ when the ratio crosses one, since $$\frac{L(n+1\mid x,p)}{L(n\mid x,p)}=\frac{1-p}{1-\dfrac{x}{n+1}}$$ is decreasing in $n\in\Bbb N$. To wit,

  • $n\mapsto x/(n+1)$ is decreasing,
  • $n\mapsto 1-x/(n+1)$ increasing,
  • $n\mapsto 1/\{1-x/(n+1)\}$ decreasing,

respectively. Hence, if $x/(n+1)>p$, i.e., if $n+1<x/p$, then ${L(n+1\mid x,p)}>{L(n\mid x,p)}$ while, if $x/(n+1)<p$, i.e., f $n+1>x/p$, then ${L(n+1|x,p)}<{L(n\mid x,p)}$. This means that $${L(\lfloor x/p\rfloor-1\mid x,p)}<{L(\lfloor,x/p\rfloor\mid x,p)}<{L(\lfloor x/p\rfloor+1\mid x,p)}$$

$\endgroup$
  • $\begingroup$ Why can't we say that the MLE of n is $ \dfrac{x}{p} -1 $ since it is the point that ratio crosses 1. But instead we choose $\bigg\lfloor\dfrac{x}{p}\bigg\rfloor$ as the MLE of n $\endgroup$ – GAGA Apr 30 at 18:05
  • $\begingroup$ Thank you , one last question. if for example $p= \dfrac{1}{2}$ thus, the by solving $ \frac{L(n+1|x,p)}{L(n|x,p)} = 1 $ we would get $n = \dfrac{x}{p} -1 =\dfrac{x}{0.5} -1 = 2*x -1 $( which is integer for every value of x ) in this case MLE of n is $n = \dfrac{x}{0.5} -1 ? or n = \dfrac{x}{0.5} $ $\endgroup$ – GAGA Apr 30 at 19:56
  • 1
    $\begingroup$ In this special case, both are MLE $\endgroup$ – Xi'an Apr 30 at 20:13
2
$\begingroup$

Binomial PMF is a discrete function of $n$, say $f(n)$, given others, i.e. $x,p$. We want to maximize it in terms of $n$. Typically, we would take the derivate and equate it to zero, but in discrete cases we shouldn't do that. This PDF is known to have its peak value(s) around its mean (not exactly but close). Its graph first increases, and then decreases. Sometimes, it stays constant for a little bit before decreasing. Therefore, the answer there considers the ratio $f(n+1)/f(n)$ and see if it is increasing or not. When the ratio is less than $1$, it means $f(n)$ is in the decreasing region, and the boundary $n$ is a candidate of ML estimate.

$\endgroup$
  • $\begingroup$ that boundary is of n is $ \dfrac{x}{p} -1 $ . But why do instead we choose $\bigg\lfloor\dfrac{x}{p}\bigg\rfloor$ as the MLE of n $\endgroup$ – GAGA Apr 30 at 18:00
  • $\begingroup$ Because $n$ must be an integer. $\endgroup$ – kjetil b halvorsen Aug 21 at 9:07
2
$\begingroup$

As Xi'an correctly points out, this is a maximisation problem over integers, not real numbers. The objective function is quasi-concave, so we can obtain the maximising value by finding the point at which the (forward) likelihood ratio first drops below one. His answer shows you how to do this, and I have nothing to add to that excellent explanation. However, it is worth noting that discrete optimisation problems like this can also be solved by solving the corresponding optimisation problem in the reals, and then considering the discrete argument points around the real optima.


Alternative optimisation method: In this particular problem it is also possible to obtain the answer via consideration of the corresponding maximisation problem over the reals. To do this, suppose we generalise the binomial likelihood function to allow non-integer values of $n$, while preserving its quasi-concavity:

$$L_x(n) = \frac{\Gamma(n+1)}{\Gamma(n-x+1)} (1-p)^{n-x} \quad \quad \quad \text{for all real } n \geqslant x.$$

This objective function is a generalisation of the binomial likelihood function ---i.e., in the special case where $n \in \mathbb{N}$ it simplifies to the binomial likelihood function you are considering. The log-likelihood function is:

$$\ell_x(n) = \ln \Gamma (n+1) - \ln \Gamma (n-x+1) + (n-x) \ln (1-p).$$

The derivatives with respect to $n$ are:

$$\begin{equation} \begin{aligned} \frac{d \ell_x}{dn}(n) &= \psi (n+1) - \psi (n-x+1) + \ln (1-p) \\[10pt] &= \sum_{i=1}^x \frac{1}{n-x+i} + \ln (1-p), \\[10pt] \frac{d^2 \ell_x}{d n^2}(n) &= - \sum_{i=1}^x \frac{1}{(n-x+i)^2} < 0. \\[10pt] \end{aligned} \end{equation}$$

(The first derivative here uses the digamma function.) We can see from this result that the log-likelihood is a strictly concave function, which means the likelihood is strictly quasi-concave. The MLE for $n$ occurs at the unique critical point of the function, which gives an implicit function for the real MLE $\hat{n}$. It is possible to establish that $x/p-1 \leqslant \hat{n} \leqslant x/p$ (see below). This narrows down the discrete MLE to be the unique point in this interval if $x/p \notin \mathbb{N}$, or one of the two boundary points if $x/p \in \mathbb{N}$. This gives an alternative derivation of the maximising value in the discrete case.


Establishing the inequalities: We have already established that the score function (first derivative of the log-likelihood) is a decreasing function. The critical point occurs at the unique point where this function crosses the zero line. To establish the inequalities, it is therefore sufficient to show that the score is non-positive at the argument value $n = x/n-1$ and non-negative at the argument value $n = x/p$.

The first of these two inequalities is established as follows:

$$\begin{equation} \begin{aligned} \frac{d \ell_x}{dn}(x/p-1) &= \sum_{i=1}^x \frac{1}{x/p-1-x+i} + \ln (1-p) \\[10pt] &= \sum_{i=1}^x \frac{p}{x(1-p)+(i-1)p} + \ln (1-p) \\[10pt] &= \sum_{i=0}^{x-1} \frac{p}{x(1-p)+ip} + \ln (1-p) \\[10pt] &\geqslant \int \limits_0^x \frac{p}{x(1-p)+ip} \ di + \ln (1-p) \\[10pt] &= \Bigg[ \ln(x(1-p)+ip) \Bigg]_{i=0}^{i=x} + \ln (1-p) \\[10pt] &= \ln(x) - \ln(x(1-p)) + \ln (1-p) \\[10pt] &= \ln(x) - \ln(x) - \ln(1-p) + \ln (1-p) = 0. \\[10pt] \end{aligned} \end{equation}$$

The second inequality is established as follows:

$$\begin{equation} \begin{aligned} \frac{d \ell_x}{dn}(x/p) &= \sum_{i=1}^x \frac{1}{x/p -x+i} + \ln (1-p) \\[10pt] &= \sum_{i=1}^x \frac{p}{x(1-p)+ip} + \ln (1-p) \\[10pt] &\leqslant \int \limits_0^x \frac{p}{x(1-p)+ip} \ di + \ln (1-p) \\[10pt] &= \Bigg[ \ln(x(1-p)+ip) \Bigg]_{i=0}^{i=x} + \ln (1-p) \\[10pt] &= \ln(x) - \ln(x(1-p)) + \ln (1-p) \\[10pt] &= \ln(x) - \ln(x) - \ln(1-p) + \ln (1-p) = 0. \\[10pt] \end{aligned} \end{equation}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.