5
$\begingroup$

From a statistical point of view, can -2LL, AIC, and BIC in the table of information criteria from SPSS output be less than zero, ie negative? In this case, how should they be interpreted? Note: These negative values were obtained while examining a two-level growth curve model to investigate volume growth trajectories over three time occasions using the SPSS software program. The histogram of the dependent variable (volume) after a two-step normalization procedure was as follows:

enter image description here

$\endgroup$
8
$\begingroup$

The bottom line is that (as Jeremy Miles says) the value of the negative log-likelihood doesn't really matter, only differences between the negative log-likelihoods. But you might still wonder why you are getting negative values.

Reproducing an answer of mine from here:

Technically, a probability cannot be >1, so a log-likelihood cannot be >0, so a negative log-likelihood cannot be negative. AIC/BIC etc. are composed of negative log-likelihoods plus positive 'penalty' terms (except in cluster analysis, where people typically flip the sign of the definitions!), so they can't be negative either.

However, there is one common case where we can get negative values for the "negative log-likelihood" function we use (which in these cases isn't exactly a negative log-likelihood). (These in turn can give rise to negative AICs (i.e., $-2\log(L)+2k<0$).)

For continuous response variables, what we are writing down is really a negative log-likelihood density function, rather than a negative log-likelihood function. For example, here's a picture of the normal density with μ=0,σ=0.1.

Normal distribution peaking around 4

You can see that the density goes above 1, which means that the log density is >0, which means that the negative log-likelihood density is negative. This will happen any time the likelihood curve is very narrow. (Mathematically, there should be an infinitesimal $dx$ term in our expressions — we typically ignore this because it doesn't affect our inferences.)


Another common scenario (but not one where I have found an example) is that we often drop the normalization constants in likelihood expressions because they're a nuisance and they don't affect inference. If we had a discrete distribution (so that the likelihood was really a probability and had to be <= 1) and the normalization constant was sufficiently small, dropping the normalization constant could in principle give us an expression that was >1. I haven't found an example where this actually happens, though ...

$\endgroup$
3
$\begingroup$

Yes.

-2 LL means -2 multiplied by the log likelihood.

AIC, BIC etc are (as far as I know) only interpreted in relation to other values from different models. An AIC of -100 doesn't mean anything on its own. It means something when a different model, using the same data, has an AIC of -90, so the difference is 10. The difference is the interesting thing.

$\endgroup$
1
  • 1
    $\begingroup$ AIC can be interpreted as twice the negative expected log-likelihood of a new data point from the same data generating process. I have written about that here and elsewhere, and others have too; see e.g. some of these threads. $\endgroup$ Aug 20 at 6:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.