3
$\begingroup$

Let $X_1, ..., X_n$ be a sample of independent, identically distributed random variables, with density

$$ f_{\theta}(x)=e^{ (\theta -x)}$$.

$x \ge \theta$, otherwise $f_\theta = 0$

The question is: Determine the maximum likelihood estimator $\hat{\theta}_n$ of $\theta$.

I don't understand this question. What exactly does $\hat{\theta}_n$ mean? Wikipedia says something about the nth order statistic:

In statistics, the kth order statistic of a statistical sample is equal to its kth-smallest value. Together with rank statistics, order statistics are among the most fundamental tools in non-parametric statistics and inference.

I tried: $$ L(\theta)=\prod_{i=1}^ne^{ (\theta -x_i)} = e^{(n\theta - \sum_{i=1}^n{x_i})} $$

What's next?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ The index in $\hat{\theta}_n$ is just notation to show that the estimator depends on the sample size $n$. You've found the likelihood function $L(\theta)$ and want to maximize it. How would you go about doing that? $\endgroup$
    – MånsT
    Oct 9, 2012 at 19:01
  • $\begingroup$ Of course I would (taking the ln of L($\theta$), differentiate with respect to $\theta$, let it equal zero, and solve for $\theta$. But I have something like an answer - it's quite unreadable- the only thing I can read is that you should do something with min{$x_1, ... , x_n$}. The final answer should be $\hat{\theta}_n=$min{$x_1, ..., x_n$}. With a note: If $\hat{\theta}$ is larger than the minimum value, the likelihood would be zero. I don't understand it :) $\endgroup$
    – Applied mathematician
    Oct 9, 2012 at 19:53
  • $\begingroup$ Hempo: it might be worth trying to set the derivative to zero that to see what happens, and why that is not the correct approach in this case. $\endgroup$
    – Henry
    Oct 9, 2012 at 19:56

1 Answer 1

Reset to default
2
$\begingroup$

Hints:

  • You have the constraint that the probability density is only positive for $x \ge \theta$ i.e. $\theta \le x$, which implies that $\hat{\theta}_n \le \min\{X_i\}$.

  • If you take the derivative of the likelihood (or the log-likelihood) with respect to $\theta$ then you ought to find the derivative is always positive

  • So?

Improve this answer
$\endgroup$
8
  • $\begingroup$ I take the log-likelihood with respect to $\theta$: L$(\theta)=\prod_{i=1}^ne^{ (\theta -x_i)} = e^{(n\theta - \sum_{i=1}^n{x_i})}$. ln(L($\theta$))= n$\theta$ - $\sum {x_i}$. Taking the derivative with respect to $\theta$ gives just ln(L($\theta$)$'$=n = 0. $\endgroup$
    – Applied mathematician
    Oct 10, 2012 at 12:05
  • $\begingroup$ Let's say the derivative of f is $n*e^{(n\theta - \sum{x_i})}=0$ Since n $\ge 1$ and $x_i \ge 0$, it follows that there is no solution for $\theta$. Whats next?? $\endgroup$
    – Applied mathematician
    Oct 10, 2012 at 18:14
  • $\begingroup$ @Hempo: so the derivative is always positive. What does that tell you about the likelihood? $\endgroup$
    – Henry
    Oct 10, 2012 at 20:53
  • $\begingroup$ I don't know? It tells me that $\theta \ge 0$ Why is the derivative always positive? $\endgroup$
    – Applied mathematician
    Oct 11, 2012 at 10:38
  • 1
    $\begingroup$ @Hempo: That is correct: the likelihood increases with $\theta$, so you want $\theta$ to be as large as possible, subject to the constraints that it is less than or equal to each $X_i$. So you want $\theta$ to be equal to the minimum of the $X_i$ for the maximum likelihood. $\endgroup$
    – Henry
    Oct 11, 2012 at 15:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.