7
$\begingroup$

$X_1, ..., X_n$ is a random sample from a population with pdf given by

$$ f(x; \mu, \lambda) = \frac{\lambda}{2}\operatorname{exp}(- \lambda |x - \mu|) $$

where $\mu \in \mathbb{R}$ is the location parameter, and $\lambda > 0$ is the scale parameter.

I'm trying to obtain the log-likehood function to this problem. I have,

$$ L(\mu, \lambda; \mathbf{x}) = \prod_{i=1}^{n} f(x_i; \mu, \lambda) = \prod_{i=1}^{n} \frac{\lambda}{2}\operatorname{exp}(- \lambda |x - \mu|) = \frac{\lambda^n}{2^n}\operatorname{exp}\Bigl(- \lambda \sum_{i=1}^{n} |x - \mu|\Bigr). $$

Let $ \ell(\mu, \lambda; \mathbf{x}) = \text{log}[L(\mu, \lambda; \mathbf{x})] $. Then we have

$$ \ell(\mu, \lambda; \mathbf{x}) = n\text{log}(\lambda/2) - \lambda \sum_{i=1}^{n} |x - \mu|. $$

Is this the correct form of the sample log-likelihood? Because if it is... then trying to maximize it results in a Hessian determinant which is negative, even before evaluation at the critical point.


Assuming the log-likehood function is correct, you'd get:

$$ \frac{\partial}{\partial \mu} \ell(\mu, \lambda; \mathbf{x}) = \lambda \sum_{i=1}^{n} \frac{x_i - \mu}{|x_i - \mu|} $$

$$ \frac{\partial}{\partial \lambda} \ell(\mu, \lambda; \mathbf{x}) = \frac{n}{\lambda} - \sum_{i=1}^{n} |x_i - \mu| $$

$$ \frac{\partial^2}{\partial \lambda^2} \ell(\mu, \lambda; \mathbf{x}) = \frac{-n}{\lambda^2}. $$

$$ \frac{\partial^2}{\partial \mu^2} \ell(\mu, \lambda; \mathbf{x}) = 0. $$

$$ \frac{\partial^2}{\partial\lambda \partial \mu} \ell(\mu, \lambda; \mathbf{x}) = \frac{\partial^2}{\partial\mu \partial \lambda} \ell(\mu, \lambda; \mathbf{x}) = \sum_{i=1}^{n} \frac{x_i - \mu}{|x_i - \mu|}. $$

Also I got the (single) critical point $ (\mu, \lambda) = (\bar{x}, n/\sum_{i=1}^{n} |x_i - \mu|) $. With this information you'd get $\text{det}(Hessian) < 0 \Rightarrow $ saddle point. So there's something wrong.

$\endgroup$

1 Answer 1

10
$\begingroup$

The loglikelihood is correct. Your problem is the derivatives.

The loglikelihood is not differentiable with respect to $\mu$ where $\mu=x_i$ (and in particular, not at the MLE). The second derivative with respect to $\mu$ is zero everywhere that it is defined. So you can't just do calculus.

For any fixed $\lambda$, the loglikelihood is piecewise linear (and concave) in $\mu$ with corners at the observations, so there is a maximum (and no other stationary points) and the maximum must be at an observation. It's fairly easy to satisfy yourself that the maximum is at the median if $n$ is odd and on the whole median interval if $n$ is even. That's true for every fixed $\lambda$ so it must be true for varying $\lambda$ as well.

Now, fixing $\mu$ at the MLE you have a differentiable one-parameter problem in $\lambda$. Solving $$\frac{\partial}{\partial \lambda} \ell(\hat\mu, \lambda ; \mathbf{x})=\frac{n}{\lambda}-\sum_{i=1}^{n}\left|x_{i}-\hat\mu\right|$$ gives you $\hat\lambda$ as the mean absolute deviation from the median as the only stationary point. You can then check the second derivative wrt $\theta$ to make sure it's a maximum. Or you can argue that the maximum must be there or at the endpoints $\lambda=0$ or $\lambda=\infty$, and it's not either of those.

$\endgroup$
2
  • $\begingroup$ I have a couple of questions. 1) You are fixing a parameter and then solving the individual partial derivatives for when they are equal to zero. Since we want the MLE for $(\mu, \lambda)$, wouldn't one have to solve both partial derivatives for 0 simultaneously? By forming a system of 2 equations and solving both. $\endgroup$
    – Sigma
    Commented Oct 24, 2020 at 15:04
  • $\begingroup$ 2) After finding the critical point/ MLE estimators $ (\tilde{x}, n/\sum_{i=1}^{n} |x_i - \mu|) $ how can I go about proving that this critical point is the global maximum? Yes the function is concave, but how could I prove that? $\endgroup$
    – Sigma
    Commented Oct 24, 2020 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.