MLE for 2 parameter exponential distribution

Let $x_1, x_2... x_n$ be a random sample from a distribution with pdf:

$$f(x;\mu,\sigma)=\frac1{\sigma}\exp\left({-\frac{x-\mu}{\sigma}}\right)\,,-\infty<\mu<\infty;\, \sigma>0;\, x\ge\mu$$

How do I find the MLE for the parameters if both parameters are unknown?

I tried using the usual MLE with likelihood function:

$$L(\mu,\sigma|x_1...x_n)=\frac{1}{\sigma^n}\exp\left({-\frac{\sum{x_i}-n\mu}{\sigma}}\right)$$ But the derivative of this with respect of $\mu$ is a dead end.

I do know that if $\sigma$ is known, the MLE for $\mu$ is $\frac{\sum{x_i}}{n}$ and if $\mu$ is known, the MLE for $\sigma$ is $\frac{\sum{x_i}-n\mu}{n}$.

Do these help? What should be the approach?

• 1. You should state the limits on the variable and the parameters; those are part of the definition of the density, and an important part of the reasoning here. 2. Not every optimization problem is solved by setting a derivative to 0. Start with a simpler problem by setting $\sigma=1$, choosing an explicit sample (e.g. 1.13, 1.56, 2.08) and draw the log-likelihood function. The required logic should be obvious – Glen_b Aug 20 '17 at 12:23
• There's additional clarification and hints for the simplified problem here – Glen_b Aug 20 '17 at 12:36
• I have provided the limits. Yes, I am aware that not all optimization can be solved using derivatives. That was how I got the MLE of $\mu$ when $\sigma$ is constant. However, I am having some difficulty on doing the same for when 2 variables $(\mu, \sigma)$ are considered. I will try the approach you stated. – user164144 Aug 20 '17 at 13:00
• When $\sigma=1$, I arrive at the conclusion that $\hat{\mu}=\bar{x}$ which I got already. I think this is the MLE for $\mu$ regardless of the value of $\sigma$ based on eyeballing the likelihood. However, that still leaves me without an estimate for $\sigma$ – user164144 Aug 20 '17 at 13:10
• I'm sorry but no, if you set $\sigma=1$ then $\hat{\mu}$ is not $\bar{x}$. If you set $\mu=0$ then $\hat{\sigma}$ would be $\bar{x}$. What did your log-likelihood look like for the specific example? When you sort that out, then try it for a known $\sigma=\sigma_0$. – Glen_b Aug 20 '17 at 21:34

Given the sample, the likelihood function is given by $$L(\mu,\sigma)=\frac{1}{\sigma^n}\exp\left[-\frac{1}{\sigma}\sum_{i=1}^n(x_i-\mu)\right]\mathbf1_{\mu\leqslant x_{(1)},\sigma>0}$$

This function is not differentiable at $\mu=x_{(1)}$, so that MLE of $\mu$ has to be found using a different argument. For fixed $\sigma$, $L(\mu,\sigma)$ is an increasing function of $\mu$ $\,\forall\,\sigma$, implying that $\hat\mu_{\text{MLE}}=X_{(1)}$.

MLE of $\sigma$ can be guessed from the first partial derivative as usual.

We have $\displaystyle\frac{\partial L(\mu,\sigma)}{\partial\sigma}=0\implies\sigma=\frac{1}{n}\sum_{i=1}^n(x_i-\mu)$.

So MLE of $\sigma$ could possibly be $\displaystyle\hat\sigma_{\text{MLE}}=\frac{1}{n}\sum_{i=1}^n(X_i-\hat\mu)=\frac{1}{n}\sum_{i=1}^n\left(X_i-X_{(1)}\right)$

The second partial derivative test fails here due to $L(\mu,\sigma)$ not being totally differentiable.

So to confirm that $(\hat\mu,\hat\sigma)$ is the MLE of $(\mu,\sigma)$, one has to verify that $L(\hat\mu,\hat\sigma)\geqslant L(\mu,\sigma)$, or somehow conclude that $\ln L(\hat\mu,\hat\sigma)\geqslant \ln L(\mu,\sigma)$ holds $\forall\,(\mu,\sigma)$.

• Would be interesting to know reason of downvote. – StubbornAtom Apr 18 at 6:13

Note that for each fixed $$\sigma > 0$$, the likelihood $$L(\mu,\sigma)$$ is an increasing function of $$\mu$$, provided that $$\mu\leq x_{(1)}$$ ($$x_{(1)}$$ being the smallest value of $$x$$). If $$\mu> x_{(1)}$$, the likelihood is $$0$$. Consequently, the MLE of $$\mu$$ is $$\hat{\mu}=x_{(1)}$$. Here is a plot of the log-likelihood for a specific example als @Glen_b suggested in the comments ($$\sigma = 1, x = \{1.13, 1.56, 2.08\}$$):

As for the MLE of $$\sigma$$, take the first derivative of the log-likelihood, set it to zero and solve for $$\sigma$$

\begin{align} \log L(x;\mu,\sigma) &=-n\log{(\sigma)}-\frac{1}{\sigma}\sum_{i=1}^{n}{(x_i-\mu)} \\ \frac{\partial \log L(x;\hat{\mu},\sigma)}{\partial \sigma}&= \frac{-n}{\sigma}+\frac{1}{\sigma^2}\sum_{i=1}^{n}{(x_i-\hat{\mu})}\\ \frac{-n}{\sigma}+\frac{1}{\sigma^2}\sum_{i=1}^{n}{(x_i-\hat{\mu})} &= 0\\ \hat{\sigma} &=\frac{1}{n}\sum_{i=1}^{n}(x_i-x_{(1)}) = \bar{x}-x_{(1)} \end{align} where $$\bar{x}$$ is the sample mean.

These results can be found in the following references.

• Rahman M & Pearson LM (2001): Estimation in two-parameter exponential distributions. Journal of Statistical Computation and Simulation, 70(4), 371-386.
• Krishnamoorthy K (2016): Handbook of Statistical Distributions with Applications. 2nd ed. Chapman and Hall/CRC.