1
$\begingroup$

For a random variable with its CDF given by $$F(x)=1-\exp(-\lambda x),$$ and its PDF given by $$f(x)=\lambda \exp(-\lambda x),$$ for $x>0$ and $\lambda >0$.

How would I write the log-likelihood function for a random sample $X_1,X_2,...,X_n$ i.i.d. Exp($\lambda$) and a maximum likelihood estimator for $\lambda$?

I know that the exponential distribution is enter image description here. I'm not quite sure where to go from there. Any help would be appreciated.

$\endgroup$
2
$\begingroup$

The likelihood is given as

$$L(\lambda,x) = L(\lambda,x_1,...,x_N) = \prod_{i=1}^N f(x_i,\lambda)$$

where the second identity use the IID assumption and with $x = (x_1,...,x_N)$. The log-likelikelihood is given as

$$l(\lambda,x) := log L(\lambda,x) = \sum_{i=1}^N \log f(x_i, \lambda),$$

where $log f(x_i,\lambda) = log \lambda - \lambda x_i$. This implies that

$$l(\lambda,x) = \sum_{i=1}^N log \lambda - \lambda x_i = N \log \lambda - \lambda \sum_{i=1}^N x_i.$$ Since we are interested in maximum a positive monotone transformation such as dividing with $N$ is fine. This gets us to

$$\frac{1}{N} l(\lambda , x) = \log \lambda - \lambda \bar x$$

differentiate and set to zero to get first order condition

$$\frac{1}{\lambda} - \bar x = 0 \Leftrightarrow \lambda = \frac{1}{\bar x}$$

Small simulation in R

lambda_0 <- 0.5
N <- 10000
x <- rexp(N, rate = lambda_0)

loglik <- function(theta)
    {
        ll <- N * log(theta) - theta*sum(x)
        return(ll)
    }

# Calculate estimate
m_x <- 1/mean(x)

# Create vector for plot of loglikelihood
t <- seq(0.5*m_x,1.5*m_x,length.out=100)

plot(t,loglik(t),type="l")
abline(v=m_x,col="red")

This will genrate this plot of loglikelihood function to see maximum ...

enter image description here

$\endgroup$
1
$\begingroup$

Any standard mathematical statistics book would give you detailed approach of how to do log likelihood. If you want better understanding of Likelihood theory then I would recommend a wonderful text In all Likelihood by Pawitan.

For your specific problem Likelihood $L$ can be written as :

$$f(\mathbf{x},\beta) = \frac{1}{\beta} \ e^{\left(\frac{-\mathbf{x}}{\beta}\right)}; \mathbf{x}>0$$

$$L(\beta,\mathbf{x}) = L(\beta,x_1,...,x_N) = \prod_{i=1}^N f(x_i,\beta)$$

$$L(\beta,\mathbf{x}) = \prod_{i=1}^N \frac{1}{\beta} \ e^{\left(\frac{-x_i}{\beta}\right)} $$

The Log likelihood $\mathscr{L} = log(L)$:

$$\mathscr{L}(\beta,\mathbf{x}) = log\left(\prod_{i=1}^N \frac{1}{\beta} \ e^{\left(\frac{-x_i}{\beta}\right)} \right)$$

So doing some algebra and applying properties of Logarithms you get:

$$\mathscr{L}(\beta,\mathbf{x}) = log\left(\prod_{i=1}^N \frac{1}{\beta} \ e^{\left(\frac{-x_i}{\beta}\right)}\right) = \sum_{i=1}^N \left( log\left(\frac{1}{\beta}\right) + log\left( e^{\left(\frac{-x_i}{\beta}\right)} \right) \right)$$

Since the first part of equation has nothing to do with summation take $log(\frac{1}{\beta})$ outside of summation.

$$\mathscr{L}(\beta,\mathbf{x}) = N \ log\left(\frac{1}{\beta}\right) + \sum_{i=1}^N \left( \frac{- x_i} {\beta} \right)$$

The above can be further simplified:

$$\mathscr{L}(\beta,\mathbf{x}) = - N \ log(\beta) + \frac{1}{\beta}\sum_{i=1}^N -x_i$$

To get the maximum likelihood, take the first partial derivative with respect to $\beta$ and equate to zero and solve for $\beta$:

$$ \frac{\partial \mathscr{L}}{\partial \beta} = \frac{\partial}{\partial \beta} \left(- N \ log(\beta) + \frac{1}{\beta}\sum_{i=1}^N -x_i \right) = 0$$

$$ \frac{\partial \mathscr{L}}{\partial \beta} = -\frac{N} {\beta} + \frac{1} {\beta^2} \sum_{i=1}^N x_i = 0$$

Now solving for $\beta$ you get:

$$\boxed{\beta = \frac{\sum_{i=1}^N x_i}{N} = \overline{\mathbf{x}}}$$

Please note that in your question $\lambda$ is parameterized as $\frac {1} {\beta}$ in the exponential distribution. Regardless of parameterization, the maximum likelihood estimator should be the same.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.