Maximum likelihood estimator of $\theta$ for uniform distribution

Question

I know that for uniformly distributed random variables $X_1,X_2,\dots,X_n$ $\in \mathcal{R}$, the p.d.f. is given by:

$f(x_i) = 1/θ$ ; if $0≤x_i≤θ$

$f(x) = 0$ ; otherwise

If the uniformly distributed random variables are arranged in the following order

$0≤X_1≤X_2≤X_3\dots ≤X_n≤θ$,

I understand that the likelihood function is given by

$L(θ)=\prod_{i=1}^{n}f(x_i)=θ^{−n}$

The log-likelihood is:

$\ln L(θ)=−n\ln(θ)$

Setting its derivative with respect to parameter $\theta$ to zero, we get:

$\frac{\mathrm d}{\mathrm d\theta}\ln L(\theta)=-n\theta$

which is $< 0$ for $θ > 0$

Hence, $L(θ)$ is a decreasing function and it is maximized at $θ = X_{(n)}$

The maximum likelihood estimate is thus

$\hat{θ} = X_{(n)}$

My question is:—what if I find the supremum to solve this?

Answer 1

The result is correct, but the reasoning is somewhat inaccurate. You need to keep track of the property that the density is zero outside $[0,\theta]$. This implies that the likelihood is zero to the left of the sample maximum, and jumps to $\theta^n$ in the maximum. It indeed decreases afterwards, so that the maximum is the MLE.

This also entails that the likelihood is not differentiable in this point, so that finding the MLE via the "canonical" route of the score function is not the way to go here.

A more detailed formal derivation is, e.g., given here

Answer 2

Assume $x\ge 0$ so that

$f(x; \theta) = \frac{1}{\theta}I(x \le \theta)$ and

$L(x; \theta) = \prod_{j=1}^J \theta^{-1}I(x_j\le \theta) = \theta^{-J}I(\max_j x_j \le \theta)$

Note that the LL is

1. Zero if $\theta$ is smaller than the largest observation. This is clearly not the maximum.
2. Decreasing in $\theta$.

So, the smallest allowed value for $\theta$ maximizes the likelihood and is given by: $\hat{\theta} = \max_j x_j$.

This makes sense: Given a uniform sample, it must be possible to generate the largest number and the most conservative estimate is that largest number. But, this underestimates the interval. Since $E[\hat{\theta}] = \frac{J}{\theta^J}\int_0^\theta y\cdot y^{J-1}\,dy=\theta\frac{J}{J+1}$ an unbiased estimate is $\hat{\theta}\frac{J+1}{J}$. This approaches the LL-estimate for large $J$.