2
$\begingroup$

This question already has an answer here:

how can I convert a negative log likelihood to likelihood between 0 and 1 ?

I use HMMs package in R and I keep getting strange results of the log likelihood for example, -48569 ! I need to understand these values so I can pick the best model. any advice will be appreciated.

thanks

|cite|improve this question
$\endgroup$

marked as duplicate by Tim♦, SmallChess, Nick Cox, gung♦, mdewey Jul 19 '17 at 10:44

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 4
    $\begingroup$ Simple answer: you cannot. First, because likelihood is not a probability. Second, if you are dealing with continuous data, then you have log-densities and (non-logged) densities are not in [0, 1] and cannot be "converted" to probabilities. $\endgroup$ – Tim♦ Jul 18 '17 at 15:02
  • $\begingroup$ thanks. I suppose there is a way to represent the values of log likelihood in much better ways. I want to train a discrete hidden Markov model not a continuous. $\endgroup$ – Amirah Jul 18 '17 at 15:10
  • $\begingroup$ Still, likelihood is not probability. If it will "look nicer" for you you can exponentiate it, but it still will not be a "probability", $\endgroup$ – Tim♦ Jul 18 '17 at 15:12
  • $\begingroup$ thanks , I have updated the question. My background is computing not statistics that's why I thought they are the same. but the purpose of my question is to get more understandable values of the likelihood. using the exponentiation gives me 0 !! $\endgroup$ – Amirah Jul 18 '17 at 15:18
  • 1
    $\begingroup$ Your question is "I have $-\log L$, how do I get $L$". This seems to be a question of basic algebraic manipulation. Where's the difficulty in undoing those two steps? $\endgroup$ – Glen_b♦ Jul 18 '17 at 23:16
4
$\begingroup$

The log likelihood is the log of the likelihood. To get the likelihood from the log likelihood just take the exponential:

$$\text{Likelihood} = e^{\text{Log Likelihood}}$$

This should result in a very small number. Instead you can get the "avg. likelihood" by line in your dataset that is easier to interpret :

$$\text{Avg. Likelihood} = e^{\frac{\text{Log Likelihood}}{\text{Number of Lines}}}$$

Now, what I'm going to say may be true for most basic models, but not for every model.

For a discrete dependent variable $Y$, the likelihood is a probability between 0 and 1. For a continuous dependant variable $Y$, it is the value of the probability density of $Y$ and may not be smaller than 1. This can be interpreted as a probability by units of $Y$.

Anyway, this probability (or density) may not have a very clear meaning. It is not, like in Bayesian analysis, the probability that the parameters are correct. It is the probability that such data is observed given the fitted model.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ thank you for your detailed answer. so what do you think if I did the following : -48569 = log(1/48569) which gives -10.79074 , then convert log to probability using 10.79074/ (1+10.79074) = 0.91 . is this correct ? $\endgroup$ – Amirah Jul 18 '17 at 15:26
  • $\begingroup$ No. The log of minus the log likelihood is nothing meaningful. $\endgroup$ – Benoit Sanchez Jul 18 '17 at 15:32
  • 2
    $\begingroup$ What your are doing this way is create an increasing function from the log likelihood to get between 0 and 1. It is nice for visualizing it as a "score". But it is not a probability. $\endgroup$ – Benoit Sanchez Jul 18 '17 at 15:39
  • $\begingroup$ For the same kind of model (same way of computing the log likelihood), then a higher log likelihood means a better fitted model. The best fitting model is the one with highest log likelihood (sign must be kept, I mean -2000 is higher than -3000). $\endgroup$ – Benoit Sanchez Jul 18 '17 at 15:50
  • $\begingroup$ many thanks and sorry I make it a long conversation! what if I keep getting better and better models ? when should I stop to avoid overfitting ? $\endgroup$ – Amirah Jul 18 '17 at 15:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.