2
$\begingroup$

how can I convert a negative log likelihood to likelihood between 0 and 1 ?

I use HMMs package in R and I keep getting strange results of the log likelihood for example, -48569 ! I need to understand these values so I can pick the best model. any advice will be appreciated.

thanks

Improve this question
$\endgroup$
5
  • 4
    $\begingroup$ Simple answer: you cannot. First, because likelihood is not a probability. Second, if you are dealing with continuous data, then you have log-densities and (non-logged) densities are not in [0, 1] and cannot be "converted" to probabilities. $\endgroup$ – Tim♦ Jul 18 '17 at 15:02
  • $\begingroup$ thanks. I suppose there is a way to represent the values of log likelihood in much better ways. I want to train a discrete hidden Markov model not a continuous. $\endgroup$ – Amirah Jul 18 '17 at 15:10
  • $\begingroup$ Still, likelihood is not probability. If it will "look nicer" for you you can exponentiate it, but it still will not be a "probability", $\endgroup$ – Tim♦ Jul 18 '17 at 15:12
  • $\begingroup$ thanks , I have updated the question. My background is computing not statistics that's why I thought they are the same. but the purpose of my question is to get more understandable values of the likelihood. using the exponentiation gives me 0 !! $\endgroup$ – Amirah Jul 18 '17 at 15:18
  • 1
    $\begingroup$ Your question is "I have $-\log L$, how do I get $L$". This seems to be a question of basic algebraic manipulation. Where's the difficulty in undoing those two steps? $\endgroup$ – Glen_b Jul 18 '17 at 23:16
5
$\begingroup$

The log likelihood is the log of the likelihood. To get the likelihood from the log likelihood just take the exponential:

$$\text{Likelihood} = e^{\text{Log Likelihood}}$$

This should result in a very small number. Instead you can get the "avg. likelihood" by line in your dataset that is easier to interpret :

$$\text{Avg. Likelihood} = e^{\frac{\text{Log Likelihood}}{\text{Number of Lines}}}$$

Now, what I'm going to say may be true for most basic models, but not for every model.

For a discrete dependent variable $Y$, the likelihood is a probability between 0 and 1. For a continuous dependant variable $Y$, it is the value of the probability density of $Y$ and may not be smaller than 1. This can be interpreted as a probability by units of $Y$.

Anyway, this probability (or density) may not have a very clear meaning. It is not, like in Bayesian analysis, the probability that the parameters are correct. It is the probability that such data is observed given the fitted model.

Improve this answer
$\endgroup$
7
  • $\begingroup$ thank you for your detailed answer. so what do you think if I did the following : -48569 = log(1/48569) which gives -10.79074 , then convert log to probability using 10.79074/ (1+10.79074) = 0.91 . is this correct ? $\endgroup$ – Amirah Jul 18 '17 at 15:26
  • $\begingroup$ No. The log of minus the log likelihood is nothing meaningful. $\endgroup$ – Benoit Sanchez Jul 18 '17 at 15:32
  • 2
    $\begingroup$ What your are doing this way is create an increasing function from the log likelihood to get between 0 and 1. It is nice for visualizing it as a "score". But it is not a probability. $\endgroup$ – Benoit Sanchez Jul 18 '17 at 15:39
  • $\begingroup$ For the same kind of model (same way of computing the log likelihood), then a higher log likelihood means a better fitted model. The best fitting model is the one with highest log likelihood (sign must be kept, I mean -2000 is higher than -3000). $\endgroup$ – Benoit Sanchez Jul 18 '17 at 15:50
  • $\begingroup$ many thanks and sorry I make it a long conversation! what if I keep getting better and better models ? when should I stop to avoid overfitting ? $\endgroup$ – Amirah Jul 18 '17 at 15:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.