2
$\begingroup$

Let $X_1, ..., X_n$ be a sample of independent, identically distributed random variables, with density

$$ f_{\theta}(x)=e^{ (\theta -x)}$$.

$x \ge \theta$, otherwise $f_\theta = 0$

The question is: Determine the maximum likelihood estimator $\hat{\theta}_n$ of $\theta$.

I don't understand this question. What exactly does $\hat{\theta}_n$ mean? Wikipedia says something about the nth order statistic:

In statistics, the kth order statistic of a statistical sample is equal to its kth-smallest value. Together with rank statistics, order statistics are among the most fundamental tools in non-parametric statistics and inference.

I tried: $$ L(\theta)=\prod_{i=1}^ne^{ (\theta -x_i)} = e^{(n\theta - \sum_{i=1}^n{x_i})} $$

What's next?

$\endgroup$
  • 1
    $\begingroup$ The index in $\hat{\theta}_n$ is just notation to show that the estimator depends on the sample size $n$. You've found the likelihood function $L(\theta)$ and want to maximize it. How would you go about doing that? $\endgroup$ – MånsT Oct 9 '12 at 19:01
  • $\begingroup$ Of course I would (taking the ln of L($\theta$), differentiate with respect to $\theta$, let it equal zero, and solve for $\theta$. But I have something like an answer - it's quite unreadable- the only thing I can read is that you should do something with min{$x_1, ... , x_n$}. The final answer should be $\hat{\theta}_n=$min{$x_1, ..., x_n$}. With a note: If $\hat{\theta}$ is larger than the minimum value, the likelihood would be zero. I don't understand it :) $\endgroup$ – Applied mathematician Oct 9 '12 at 19:53
  • $\begingroup$ Hempo: it might be worth trying to set the derivative to zero that to see what happens, and why that is not the correct approach in this case. $\endgroup$ – Henry Oct 9 '12 at 19:56
2
$\begingroup$

Hints:

  • You have the constraint that the probability density is only positive for $x \ge \theta$ i.e. $\theta \le x$, which implies that $\hat{\theta}_n \le \min\{X_i\}$.

  • If you take the derivative of the likelihood (or the log-likelihood) with respect to $\theta$ then you ought to find the derivative is always positive

  • So?

$\endgroup$
  • $\begingroup$ I take the log-likelihood with respect to $\theta$: L$(\theta)=\prod_{i=1}^ne^{ (\theta -x_i)} = e^{(n\theta - \sum_{i=1}^n{x_i})}$. ln(L($\theta$))= n$\theta$ - $\sum {x_i}$. Taking the derivative with respect to $\theta$ gives just ln(L($\theta$)$'$=n = 0. $\endgroup$ – Applied mathematician Oct 10 '12 at 12:05
  • $\begingroup$ Let's say the derivative of f is $n*e^{(n\theta - \sum{x_i})}=0$ Since n $\ge 1$ and $x_i \ge 0$, it follows that there is no solution for $\theta$. Whats next?? $\endgroup$ – Applied mathematician Oct 10 '12 at 18:14
  • $\begingroup$ @Hempo: so the derivative is always positive. What does that tell you about the likelihood? $\endgroup$ – Henry Oct 10 '12 at 20:53
  • $\begingroup$ I don't know? It tells me that $\theta \ge 0$ Why is the derivative always positive? $\endgroup$ – Applied mathematician Oct 11 '12 at 10:38
  • $\begingroup$ I guess I know it: the derivative is always positive, meaning that you need the greatest possible value of $\theta$, that is min($x_i$)? $\endgroup$ – Applied mathematician Oct 11 '12 at 13:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.