# Maximum likelihood estimation and the n-th order statistic

Let $X_1, ..., X_n$ be a sample of independent, identically distributed random variables, with density

$$f_{\theta}(x)=e^{ (\theta -x)}$$.

$x \ge \theta$, otherwise $f_\theta = 0$

The question is: Determine the maximum likelihood estimator $\hat{\theta}_n$ of $\theta$.

I don't understand this question. What exactly does $\hat{\theta}_n$ mean? Wikipedia says something about the nth order statistic:

In statistics, the kth order statistic of a statistical sample is equal to its kth-smallest value. Together with rank statistics, order statistics are among the most fundamental tools in non-parametric statistics and inference.

I tried: $$L(\theta)=\prod_{i=1}^ne^{ (\theta -x_i)} = e^{(n\theta - \sum_{i=1}^n{x_i})}$$

What's next?

• The index in $\hat{\theta}_n$ is just notation to show that the estimator depends on the sample size $n$. You've found the likelihood function $L(\theta)$ and want to maximize it. How would you go about doing that? Oct 9 '12 at 19:01
• Of course I would (taking the ln of L($\theta$), differentiate with respect to $\theta$, let it equal zero, and solve for $\theta$. But I have something like an answer - it's quite unreadable- the only thing I can read is that you should do something with min{$x_1, ... , x_n$}. The final answer should be $\hat{\theta}_n=$min{$x_1, ..., x_n$}. With a note: If $\hat{\theta}$ is larger than the minimum value, the likelihood would be zero. I don't understand it :) Oct 9 '12 at 19:53
• Hempo: it might be worth trying to set the derivative to zero that to see what happens, and why that is not the correct approach in this case. Oct 9 '12 at 19:56

Hints:

• You have the constraint that the probability density is only positive for $x \ge \theta$ i.e. $\theta \le x$, which implies that $\hat{\theta}_n \le \min\{X_i\}$.

• If you take the derivative of the likelihood (or the log-likelihood) with respect to $\theta$ then you ought to find the derivative is always positive

• So?

• I take the log-likelihood with respect to $\theta$: L$(\theta)=\prod_{i=1}^ne^{ (\theta -x_i)} = e^{(n\theta - \sum_{i=1}^n{x_i})}$. ln(L($\theta$))= n$\theta$ - $\sum {x_i}$. Taking the derivative with respect to $\theta$ gives just ln(L($\theta$)$'$=n = 0. Oct 10 '12 at 12:05
• Let's say the derivative of f is $n*e^{(n\theta - \sum{x_i})}=0$ Since n $\ge 1$ and $x_i \ge 0$, it follows that there is no solution for $\theta$. Whats next?? Oct 10 '12 at 18:14
• @Hempo: so the derivative is always positive. What does that tell you about the likelihood? Oct 10 '12 at 20:53
• I don't know? It tells me that $\theta \ge 0$ Why is the derivative always positive? Oct 11 '12 at 10:38
• @Hempo: That is correct: the likelihood increases with $\theta$, so you want $\theta$ to be as large as possible, subject to the constraints that it is less than or equal to each $X_i$. So you want $\theta$ to be equal to the minimum of the $X_i$ for the maximum likelihood. Oct 11 '12 at 15:56