3
$\begingroup$

I know that for uniformly distributed random variables $X_1,X_2,\dots,X_n$ $\in \mathcal{R}$, the p.d.f. is given by:

$f(x_i) = 1/θ$ ; if $0≤x_i≤θ$

$f(x) = 0$ ; otherwise

If the uniformly distributed random variables are arranged in the following order

$0≤X_1≤X_2≤X_3\dots ≤X_n≤θ$,

I understand that the likelihood function is given by

$L(θ)=\prod_{i=1}^{n}f(x_i)=θ^{−n}$

The log-likelihood is:

$\ln L(θ)=−n\ln(θ)$

Setting its derivative with respect to parameter $\theta$ to zero, we get:

$\frac{\mathrm d}{\mathrm d\theta}\ln L(\theta)=-n\theta$

which is $< 0$ for $θ > 0$

Hence, $L(θ)$ is a decreasing function and it is maximized at $θ = X_{(n)}$

The maximum likelihood estimate is thus

$\hat{θ} = X_{(n)}$

My question is:—what if I find the supremum to solve this?

$\endgroup$
5
  • 1
    $\begingroup$ You may want to have a look at formatting your questions in LaTeX, which is possible quite easily here on CV. $\endgroup$ Jun 25, 2020 at 11:18
  • 4
    $\begingroup$ What is your question? $\endgroup$
    – whuber
    Jun 25, 2020 at 13:34
  • $\begingroup$ Please use MathJax for typesetting. $\endgroup$ Jun 25, 2020 at 14:45
  • 1
    $\begingroup$ The likelihood function is given by: $$L(θ)=\prod_{i=1}^{n}f(x_i)=θ^{−n}$$ this is incorrect and should be $$L(θ)= \begin{cases} \prod_{i=1}^{n}f(x_i)=θ^{−n} & \text{if} &\forall i : 0<x_i<\theta \\ 0 & \text{else}\\ \end{cases} $$ $\endgroup$ Jun 26, 2020 at 10:43
  • 2
    $\begingroup$ "and it is maximized at $\theta = X_{(n)}$" this has not to do with the derivative of the likelihood, but with the likelihood being zero if $\theta < x_{(n)}$. $\endgroup$ Jun 26, 2020 at 10:52

2 Answers 2

8
$\begingroup$

The result is correct, but the reasoning is somewhat inaccurate. You need to keep track of the property that the density is zero outside $[0,\theta]$. This implies that the likelihood is zero to the left of the sample maximum, and jumps to $\theta^n$ in the maximum. It indeed decreases afterwards, so that the maximum is the MLE.

This also entails that the likelihood is not differentiable in this point, so that finding the MLE via the "canonical" route of the score function is not the way to go here.

A more detailed formal derivation is, e.g., given here

$\endgroup$
3
  • $\begingroup$ But shouldn't MLE be X(n)? Ain't this theory wrong? How about using supremum to find it from the interval? $\endgroup$ Jun 25, 2020 at 11:19
  • $\begingroup$ It would be useful to define your notation, both $Xn$, $X(n)$ show up. The MLE is the sample maximum $\endgroup$ Jun 25, 2020 at 11:21
  • $\begingroup$ I got it. Thanks~ $\endgroup$ Jun 25, 2020 at 11:22
2
$\begingroup$

Assume $x\ge 0$ so that

$f(x; \theta) = \frac{1}{\theta}I(x \le \theta)$ and

$L(x; \theta) = \prod_{j=1}^J \theta^{-1}I(x_j\le \theta) = \theta^{-J}I(\max_j x_j \le \theta)$

Note that the LL is

  1. Zero if $\theta$ is smaller than the largest observation. This is clearly not the maximum.
  2. Decreasing in $\theta$.

So, the smallest allowed value for $\theta$ maximizes the likelihood and is given by: $\hat{\theta} = \max_j x_j$.

This makes sense: Given a uniform sample, it must be possible to generate the largest number and the most conservative estimate is that largest number. But, this underestimates the interval. Since $E[\hat{\theta}] = \frac{J}{\theta^J}\int_0^\theta y\cdot y^{J-1}\,dy=\theta\frac{J}{J+1}$ an unbiased estimate is $\hat{\theta}\frac{J+1}{J}$. This approaches the LL-estimate for large $J$.

$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.