Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

If $X$ is taken from a discrete uniform distribution, {1,2,.....,N}, why is the MLE of N=X? I can give an intuitive argument in that since you are only viewing one observation, it is the largest value and hence is the maximum.

But, is there a nice mathematical derivation via argmax or anything like that? Thanks!

share|improve this question
    
Please also have a look at this video which explains this very well using the German tank problem. Link to the video –  Luca Feb 17 at 21:19

3 Answers 3

up vote 4 down vote accepted

Looking at the general case of having a sample of $m$ realizations of an i.i.d. sample, the joint density of the sample is

$$f(\mathbf X) = \prod_{i=1}^m \frac 1n \cdot \mathbf I\Big\{X_i\in \{1,...,n\}\Big\} $$

where $\mathbf I\{\}$ is the indicator function. Using the properties of the indicator function, and treating the joint density as a likelihood function of the unknown parameter $n$ given the actual realization of the sample, we have

$$ L(n \mid \mathbf x) = \frac 1{n^m} \cdot \min_i\Big(\mathbf I\Big\{x_i \le n\Big\}\Big)$$

Now, if we have overlooked the existence of the indicator function, then the term $\frac 1{n^m}$ is decreasing in $n$ - so the value of $n$ that would maximize the likelihood would be the smallest possible $n$, i.e $n=1$. But the existence of the (minimum of) the $m$ indicator functions tells us that if even one $x$-realization is smaller than the chosen value of $n$, then the indicator function for this $x$-realization will equal zero, hence the minimum of the $m$ indicator functions will equal zero, hence the likelihood will equal zero. So we need to choose $\hat n$ so as all realizations of the sample are equal or smaller than it... so why not choose an arbitrary large value? Because the further away we move from $\hat n =1$ the smaller the value of the likelihood becomes. So we want to move away the less possible: this means that we choose $\hat n = \max_i\{x_i\}$ which is the argmax of the likelihood given the constraint, as this constraint is represented by the indicator function, because it reduces the value of the likelihood no more than needed in order to satisfy the constraint.

Obviously the above holds for $m=1$ also.

share|improve this answer

It's pretty obvious mathematically. I tend to think it's less so intuitively. Suppose the true max is Y. Then Y has to be $\ge$ X. But then the prob of observing X is exactly 1/Y. How do you think 1/Y is maximized under the lower bound constraint?

share|improve this answer

The density function of the discrete uniform distribution is

$f(x)= 1/ N \quad for \ x=x_i \ with \ i \in [1,\dots, N] \\ f(x)= 0 \quad else.$

Hence, the likelihood function is $L(X_1,\dots,X_n;N) = N^{-n}$.

Thus, the MLE is $\hat N = arg\ maxL(X_1,\dots,X_n;N)=max(X_1,\dots,X_n).$

share|improve this answer
1  
The question is about the discrete uniform on $1,2,...,N$, rather than the continuous on $[0,\theta]$; your answer would need to be modified slightly to cover the case in the question. –  Glen_b Feb 17 at 21:25
    
I made some edits and hope it's okay now. –  random_guy Feb 17 at 21:54

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.