Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I would like to ask a question on a practice problem from a textbook.

The practice problem is about finding estimators of $\theta$, first by using method of moments and then by using a maximum likelihood function, for the following pdf:

$f(x) = \begin{cases} e^{-(x-\theta)}, &\mbox{if }\: x \ge \theta \\ 0, & \mbox{otherwise}. \end{cases} $

To obtain $\hat{\theta}$ via method of moments, I set $\bar{X} = E[X] = \int^{\infty}_\theta x e^{-(\theta -x)}dx = \theta+1,$ giving $\hat{\theta}$ = $\bar{X}-1$.

I am stuck trying to obtain an estimator using a maximum likelihood function. The practice problem says to show that the MLE of $\theta$ is $min(X_i)$.

I set up the following:

$L(\theta ; X_i)=\prod_{i=1}^n e^{-(x_i - \theta)}$

$=e^{-\sum_{i=1}^n (x_i - \theta)}$

Based on a general graph of $y=e^{-x}$, wouldn't one maximise the value of $y$ by minimising the value of $x$? So does that imply we need to minimize $\sum_{i=1}^n (x_i - \theta)$? I am confused here, and would appreciate any tips on how to proceed.

Apologies for any atrocious maths on my part. And thanks in advance.

share|improve this question
up vote 3 down vote accepted

Your likelihood is incomplete, which is why you are confused. The density of $X$ can be written in terms of indicator functions, rather than piecewise: $$f(x \mid \theta) = e^{-(x-\theta)}{\mathbf 1}(x \ge \theta),$$ where the term $\mathbf 1 (x \ge \theta)$ equals $1$ if $x \ge \theta$, and $0$ otherwise. So the likelihood of $\theta$ for a given sample $\boldsymbol x = (x_1, x_2, \ldots, x_n)$ is now $$L(\theta \mid \boldsymbol x) = \prod_{i=1}^n e^{-(x_i - \theta)} {\mathbf 1}(x_i \ge \theta) = e^{-n(\bar x - \theta)} \prod_{i=1}^n {\mathbf 1}(x_i \ge \theta).$$ This last product can be further simplified, by observing that in order for the product to equal $1$, every single term must also be $1$: if any $x_i < \theta$, then the product is zero. That is to say, the smallest $x_i$ must be at least $\theta$, or $$\prod_{i=1}^n {\mathbf 1}(x_i \ge \theta) = {\mathbf 1}(x_{(1)} \ge \theta),$$ where $x_{(1)} = \min_i x_i$ is the first order statistic. Consequently, the log-likelihood is $$\ell(\theta \mid \boldsymbol x) = -n(\bar x - \theta) + \log {\mathbf 1}(x_{(1)} \ge \theta).$$ At this point, we should emphasize that $L$ and $\ell$ are regarded as functions of $\theta$ for a given sample $\boldsymbol x$. So to maximize $L$, we need only consider the global maximum of $\ell$ on $\theta \in (-\infty, x_{(1)}]$. In other words, we cannot choose a $\theta$ that is bigger than the smallest observation in the sample. Under such a condition, the term $\log {\mathbf 1}(x_{(1)} \ge \theta)$ is always zero. Then we note that $-n(\bar x - \theta)$ is a strictly increasing function of $\theta$. So $\ell$ attains a maximum at $\hat \theta = x_{(1)}$, and this is the MLE.

share|improve this answer
Great, thanks for such a detailed answer. – Uday Pramod Mar 21 '14 at 9:02
heropup Could you take a look at the guidelines for answering self-study questions, please - specifically the request to lean toward 'hints' rather than detailed answers. – Glen_b Mar 21 '14 at 9:20

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.