8
$\begingroup$

If $X$ is taken from a discrete uniform distribution, {1,2,.....,N}, why is the MLE of N=X? I can give an intuitive argument in that since you are only viewing one observation, it is the largest value and hence is the maximum.

But, is there a nice mathematical derivation via argmax or anything like that? Thanks!

$\endgroup$
1
  • $\begingroup$ Please also have a look at this video which explains this very well using the German tank problem. Link to the video $\endgroup$
    – Luca
    Feb 17, 2014 at 21:19

3 Answers 3

10
$\begingroup$

Looking at the general case of having a sample of $m$ realizations of an i.i.d. sample, the joint density of the sample is

$$f(\mathbf X) = \prod_{i=1}^m \frac 1n \cdot \mathbf I\Big\{X_i\in \{1,...,n\}\Big\} $$

where $\mathbf I\{\}$ is the indicator function. Using the properties of the indicator function, and treating the joint density as a likelihood function of the unknown parameter $n$ given the actual realization of the sample, we have

$$ L(n \mid \mathbf x) = \frac 1{n^m} \cdot \min_i\Big(\mathbf I\Big\{x_i \le n\Big\}\Big)$$

Now, if we have overlooked the existence of the indicator function, then the term $\frac 1{n^m}$ is decreasing in $n$ - so the value of $n$ that would maximize the likelihood would be the smallest possible $n$, i.e $n=1$. But the existence of the (minimum of) the $m$ indicator functions tells us that if even one $x$-realization is larger than the chosen value of $n$, then the indicator function for this $x$-realization will equal zero, hence the minimum of the $m$ indicator functions will equal zero, hence the likelihood will equal zero. So we need to choose $\hat n$ so as all realizations of the sample are equal or smaller than it... so why not choose an arbitrary large value? Because the further away we move from $\hat n =1$ the smaller the value of the likelihood becomes. So we want to move away the less possible: this means that we choose $\hat n = \max_i\{x_i\}$ which is the argmax of the likelihood given the constraint, as this constraint is represented by the indicator function, because it reduces the value of the likelihood no more than needed in order to satisfy the constraint.

Obviously the above holds for $m=1$ also.

RESPONSE TO COMMENT 2022-05-28 (user @user1916067) Suppose we have a sample of 3 observations,say, $\{3, 4, 5\}$ and so $m=3$. Using the expression of $L(n | \mathbf x)$, compute the value of the likelihood for the following candidate MLEs

  1. A value smaller than the realized values, say $\hat n = 2$
  2. The values from the observed sample, $\hat n = \{3, 4, 5\}$
  3. A value larger than the realized values, say $\hat n = 6$

You will find that the value of the likelihood for $\hat n = \{2,3,4\}$ is zero, and the value for $\hat n = 6$ is smaller than its value when $\hat n = 5 = \max x_i$.

$\endgroup$
2
  • $\begingroup$ I was able to follow the argument up to the where you choose $n^{^} = max_{i} {x_i}$. Can you explain that point in another way? Thanks. $\endgroup$ May 27, 2022 at 7:54
  • $\begingroup$ @user1916067 I am not sure I can. I added a numerical example for you to work out. $\endgroup$ May 28, 2022 at 11:11
6
$\begingroup$

It's pretty obvious mathematically. I tend to think it's less so intuitively. Suppose the true max is Y. Then Y has to be $\ge$ X. But then the prob of observing X is exactly 1/Y. How do you think 1/Y is maximized under the lower bound constraint?

$\endgroup$
1
  • $\begingroup$ this one makes so much sense! $\endgroup$ Oct 7, 2021 at 21:55
6
$\begingroup$

The density function of the discrete uniform distribution is

$f(x)= 1/ N \quad for \ x=x_i \ with \ i \in [1,\dots, N] \\ f(x)= 0 \quad else.$

Hence, the likelihood function is $L(X_1,\dots,X_n;N) = N^{-n}$.

Thus, the MLE is $\hat N = arg\ maxL(X_1,\dots,X_n;N)=max(X_1,\dots,X_n).$

$\endgroup$
2
  • 1
    $\begingroup$ The question is about the discrete uniform on $1,2,...,N$, rather than the continuous on $[0,\theta]$; your answer would need to be modified slightly to cover the case in the question. $\endgroup$
    – Glen_b
    Feb 17, 2014 at 21:25
  • $\begingroup$ I made some edits and hope it's okay now. $\endgroup$
    – random_guy
    Feb 17, 2014 at 21:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.