Method of moments and maximum likelihood problem

Question

I would like to ask a question on a practice problem from a textbook.

The practice problem is about finding estimators of $\theta$, first by using method of moments and then by using a maximum likelihood function, for the following pdf:

$f(x) = \begin{cases} e^{-(x-\theta)}, &\mbox{if }\: x \ge \theta \\ 0, & \mbox{otherwise}. \end{cases} $

To obtain $\hat{\theta}$ via method of moments, I set $\bar{X} = E[X] = \int^{\infty}_\theta x e^{-(\theta -x)}dx = \theta+1,$ giving $\hat{\theta}$ = $\bar{X}-1$.

I am stuck trying to obtain an estimator using a maximum likelihood function. The practice problem says to show that the MLE of $\theta$ is $min(X_i)$.

I set up the following:

$L(\theta ; X_i)=\prod_{i=1}^n e^{-(x_i - \theta)}$

$=e^{-\sum_{i=1}^n (x_i - \theta)}$

Based on a general graph of $y=e^{-x}$, wouldn't one maximise the value of $y$ by minimising the value of $x$? So does that imply we need to minimize $\sum_{i=1}^n (x_i - \theta)$? I am confused here, and would appreciate any tips on how to proceed.

Apologies for any atrocious maths on my part. And thanks in advance.

Answer 1

Your likelihood is incomplete, which is why you are confused. The density of $X$ can be written in terms of indicator functions, rather than piecewise: $$f(x \mid \theta) = e^{-(x-\theta)}{\mathbf 1}(x \ge \theta),$$ where the term $\mathbf 1 (x \ge \theta)$ equals $1$ if $x \ge \theta$, and $0$ otherwise. So the likelihood of $\theta$ for a given sample $\boldsymbol x = (x_1, x_2, \ldots, x_n)$ is now $$L(\theta \mid \boldsymbol x) = \prod_{i=1}^n e^{-(x_i - \theta)} {\mathbf 1}(x_i \ge \theta) = e^{-n(\bar x - \theta)} \prod_{i=1}^n {\mathbf 1}(x_i \ge \theta).$$ This last product can be further simplified, by observing that in order for the product to equal $1$, every single term must also be $1$: if any $x_i < \theta$, then the product is zero. That is to say, the smallest $x_i$ must be at least $\theta$, or $$\prod_{i=1}^n {\mathbf 1}(x_i \ge \theta) = {\mathbf 1}(x_{(1)} \ge \theta),$$ where $x_{(1)} = \min_i x_i$ is the first order statistic. Consequently, the log-likelihood is $$\ell(\theta \mid \boldsymbol x) = -n(\bar x - \theta) + \log {\mathbf 1}(x_{(1)} \ge \theta).$$ At this point, we should emphasize that $L$ and $\ell$ are regarded as functions of $\theta$ for a given sample $\boldsymbol x$. So to maximize $L$, we need only consider the global maximum of $\ell$ on $\theta \in (-\infty, x_{(1)}]$. In other words, we cannot choose a $\theta$ that is bigger than the smallest observation in the sample. Under such a condition, the term $\log {\mathbf 1}(x_{(1)} \ge \theta)$ is always zero. Then we note that $-n(\bar x - \theta)$ is a strictly increasing function of $\theta$. So $\ell$ attains a maximum at $\hat \theta = x_{(1)}$, and this is the MLE.