I would like to ask a question on a practice problem from a textbook.

The practice problem is about finding estimators of $\theta$, first by using method of moments and then by using a maximum likelihood function, for the following pdf:

$f(x) = \begin{cases} e^{-(x-\theta)}, &\mbox{if }\: x \ge \theta \\ 0, & \mbox{otherwise}. \end{cases} $

To obtain $\hat{\theta}$ via method of moments, I set $\bar{X} = E[X] = \int^{\infty}_\theta x e^{-(\theta -x)}dx = \theta+1,$ giving $\hat{\theta}$ = $\bar{X}-1$.

I am stuck trying to obtain an estimator using a maximum likelihood function. The practice problem says to show that the MLE of $\theta$ is $min(X_i)$.

I set up the following:

$L(\theta ; X_i)=\prod_{i=1}^n e^{-(x_i - \theta)}$

$=e^{-\sum_{i=1}^n (x_i - \theta)}$

Based on a general graph of $y=e^{-x}$, wouldn't one maximise the value of $y$ by minimising the value of $x$? So does that imply we need to minimize $\sum_{i=1}^n (x_i - \theta)$? I am confused here, and would appreciate any tips on how to proceed.

Apologies for any atrocious maths on my part. And thanks in advance.


1 Answer 1


Your likelihood is incomplete, which is why you are confused. The density of $X$ can be written in terms of indicator functions, rather than piecewise: $$f(x \mid \theta) = e^{-(x-\theta)}{\mathbf 1}(x \ge \theta),$$ where the term $\mathbf 1 (x \ge \theta)$ equals $1$ if $x \ge \theta$, and $0$ otherwise. So the likelihood of $\theta$ for a given sample $\boldsymbol x = (x_1, x_2, \ldots, x_n)$ is now $$L(\theta \mid \boldsymbol x) = \prod_{i=1}^n e^{-(x_i - \theta)} {\mathbf 1}(x_i \ge \theta) = e^{-n(\bar x - \theta)} \prod_{i=1}^n {\mathbf 1}(x_i \ge \theta).$$ This last product can be further simplified, by observing that in order for the product to equal $1$, every single term must also be $1$: if any $x_i < \theta$, then the product is zero. That is to say, the smallest $x_i$ must be at least $\theta$, or $$\prod_{i=1}^n {\mathbf 1}(x_i \ge \theta) = {\mathbf 1}(x_{(1)} \ge \theta),$$ where $x_{(1)} = \min_i x_i$ is the first order statistic. Consequently, the log-likelihood is $$\ell(\theta \mid \boldsymbol x) = -n(\bar x - \theta) + \log {\mathbf 1}(x_{(1)} \ge \theta).$$ At this point, we should emphasize that $L$ and $\ell$ are regarded as functions of $\theta$ for a given sample $\boldsymbol x$. So to maximize $L$, we need only consider the global maximum of $\ell$ on $\theta \in (-\infty, x_{(1)}]$. In other words, we cannot choose a $\theta$ that is bigger than the smallest observation in the sample. Under such a condition, the term $\log {\mathbf 1}(x_{(1)} \ge \theta)$ is always zero. Then we note that $-n(\bar x - \theta)$ is a strictly increasing function of $\theta$. So $\ell$ attains a maximum at $\hat \theta = x_{(1)}$, and this is the MLE.

  • 1
    $\begingroup$ heropup Could you take a look at the guidelines for answering self-study questions, please - specifically the request to lean toward 'hints' rather than detailed answers. $\endgroup$
    – Glen_b
    Commented Mar 21, 2014 at 9:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.