13
$\begingroup$

I have difficulty interpreting some results. I am doing an hierarchical related regression with ecoreg. If I enter the code I receive output with odds ratios, confidence ratios and a 2x maximized log likelihood.

However, I do not fully understand how to interpret the 2x maximized log likelihood. As far as I know log likelihood is used as a convenient way of calculating a likelihood and it calculates the value of the parameters based on the outcomes. But I do not understand if a higher or lower value is better. I looked at several online sources e.g. https://stackoverflow.com/questions/2343093/what-is-log-likelihood, but I am still stuck.

Below the outcome I receive:

Call:
eco(formula = cbind(y, N) ~ deprivation + meanIncome, binary = ~fracSmoke + 
    soclass, data = dfAggPlus, cross = cross)

Aggregate-level odds ratios: 
                   OR        l95        u95
(Intercept) 0.0510475 0.03837276 0.06790878
deprivation 0.9859936 0.88421991 1.09948134
meanIncome  1.0689951 0.95574925 1.19565924

Individual-level odds ratios:
                OR       l95      u95
fracSmoke 3.124053 2.0761956 4.700765
soclass   1.001050 0.9930815 1.009083

-2 x log-likelihood:  237.4882 

So, how should I interpreted a value of 237.4882 compared to an outcome of 206 or 1083? Help is much appreciated!

$\endgroup$
5
  • $\begingroup$ What exactly is unclear for you? $\endgroup$
    – Tim
    May 25, 2016 at 16:08
  • $\begingroup$ Well, I want to understand if a higher log likelihood means the outcome is more reliable or for example less reliable. Furthermore I want to know how I should interpret the differences between several outcomes (e.g. 206 237 or 1083) $\endgroup$
    – Keizer
    May 26, 2016 at 14:08
  • $\begingroup$ Possible duplicate of Maximum Likelihood Estimation (MLE) in layman terms $\endgroup$
    – Tim
    May 26, 2016 at 18:09
  • $\begingroup$ I marked your question as a duplicate of another, more general, question that asks what is maximum likelihood estimation -- check it. $\endgroup$
    – Tim
    May 26, 2016 at 18:11
  • $\begingroup$ It seems it's used as a deviance. See <en.wikipedia.org/wiki/Deviance_information_criterion> $\endgroup$
    – pglpm
    Aug 24, 2019 at 23:03

2 Answers 2

2
$\begingroup$

Maximum likelihood estimation works by trying to maximize the likelihood. As the log function is strictly increasing, maximizing the log-likelihood will maximize the likelihood. We do this as the likelihood is a product of very small numbers and tends to underflow on computers rather quickly. The log-likelihood is the summation of negative numbers, which doesn't overflow except in pathological cases. Multiplying by -2 (and the 2 comes from Akaike and linear regression) turns the maximization problem into a minimization problem. So MLE implementations of regression can be considered to work work by minimizing the negative log-likelihood (NLL).

Therefore, the lower the NLL, the better the fit. However, this leads to overfitting, since adding more parameters will provide a better fit to the observed data making the variance of the fit much larger: it will fit more poorly on new data. The information criteria measures like AIC, AICc, etc. have other terms related the number of parameters, data, or both that helps ameliorate the tendency to use over-parametrized distributions.

$\endgroup$
0
$\begingroup$

To get at least some meaning out of the likelihood L, you could remember that for fix sample count N the maximum log-likelihood for a certain distribution model depends mainly on the scale. For given variance, the normal distribution has the highest value. To get some insight I would divide logL by N, and then maybe also do a correction for scale. If your data fits better to a uniform distribution, then it would be better to use the uniform likelihood as maximum entropy function for given range as a kind of reference. Another general reference value might be for continous case to use a KDE fit, and to calculate the L for this. However, whatever you do L is harder to interpret than e. g. the KS value or the rms error.
If you take another model and get a higher L, then it does not mean the model is better, because maybe you are in an overfitting situation. To include this use the AIC value. Here lower is better, and again you may use a normal distribution as "reference".

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.