# Maximum likelihood estimation and the n-th order statistic

Let $X_1, ..., X_n$ be a sample of independent, identically distributed random variables, with density

$$f_{\theta}(x)=e^{ (\theta -x)}$$.

$x \ge \theta$, otherwise $f_\theta = 0$

The question is: Determine the maximum likelihood estimator $\hat{\theta}_n$ of $\theta$.

I don't understand this question. What exactly does $\hat{\theta}_n$ mean? Wikipedia says something about the nth order statistic:

In statistics, the kth order statistic of a statistical sample is equal to its kth-smallest value. Together with rank statistics, order statistics are among the most fundamental tools in non-parametric statistics and inference.

I tried: $$L(\theta)=\prod_{i=1}^ne^{ (\theta -x_i)} = e^{(n\theta - \sum_{i=1}^n{x_i})}$$

What's next?

• The index in $\hat{\theta}_n$ is just notation to show that the estimator depends on the sample size $n$. You've found the likelihood function $L(\theta)$ and want to maximize it. How would you go about doing that? Oct 9, 2012 at 19:01
• Of course I would (taking the ln of L($\theta$), differentiate with respect to $\theta$, let it equal zero, and solve for $\theta$. But I have something like an answer - it's quite unreadable- the only thing I can read is that you should do something with min{$x_1, ... , x_n$}. The final answer should be $\hat{\theta}_n=$min{$x_1, ..., x_n$}. With a note: If $\hat{\theta}$ is larger than the minimum value, the likelihood would be zero. I don't understand it :) Oct 9, 2012 at 19:53
• Hempo: it might be worth trying to set the derivative to zero that to see what happens, and why that is not the correct approach in this case. Oct 9, 2012 at 19:56

Hints:

• You have the constraint that the probability density is only positive for $x \ge \theta$ i.e. $\theta \le x$, which implies that $\hat{\theta}_n \le \min\{X_i\}$.

• If you take the derivative of the likelihood (or the log-likelihood) with respect to $\theta$ then you ought to find the derivative is always positive

• So?

• I take the log-likelihood with respect to $\theta$: L$(\theta)=\prod_{i=1}^ne^{ (\theta -x_i)} = e^{(n\theta - \sum_{i=1}^n{x_i})}$. ln(L($\theta$))= n$\theta$ - $\sum {x_i}$. Taking the derivative with respect to $\theta$ gives just ln(L($\theta$)$'$=n = 0. Oct 10, 2012 at 12:05
• Let's say the derivative of f is $n*e^{(n\theta - \sum{x_i})}=0$ Since n $\ge 1$ and $x_i \ge 0$, it follows that there is no solution for $\theta$. Whats next?? Oct 10, 2012 at 18:14
• @Hempo: so the derivative is always positive. What does that tell you about the likelihood? Oct 10, 2012 at 20:53
• I don't know? It tells me that $\theta \ge 0$ Why is the derivative always positive? Oct 11, 2012 at 10:38
• @Hempo: That is correct: the likelihood increases with $\theta$, so you want $\theta$ to be as large as possible, subject to the constraints that it is less than or equal to each $X_i$. So you want $\theta$ to be equal to the minimum of the $X_i$ for the maximum likelihood. Oct 11, 2012 at 15:56