Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have tried calculating the AIC of a linear regression in R but without using the AIC function, like this:

lm_mtcars <- lm(mpg ~ drat, mtcars)

nrow(mtcars)*(log((sum(lm_mtcars$residuals^2)/nrow(mtcars))))+(length(lm_mtcars$coefficients)*2)
[1] 97.98786

However, AIC gives a different value:

AIC(lm_mtcars)
[1] 190.7999

Could somebody tell me what I'm doing wrong?

share|improve this question
4  
(without checking your answer yet): You're not necessarily doing anything wrong, since likelihood is actually only defined up to a multiplicative constant; two people can calculate log-likelihood and get different numbers (but differences in log-likelihood is the same). –  Glen_b Feb 20 at 20:26
    
Hong Oois answer is related to this question, I think. The formula that the function AIC uses is -2*as.numeric(logLik(lm_mtcars))+2*(length(lm_mtcars$coefficients)+1). –  COOLSerdash Feb 20 at 20:27
    
luciano: The "+1" in that formula @COOLSerdash points to arises from the variance parameter term. Note also that the function logLik says that for lm models it includes 'all constants' ... so there'll be a log(2*pi) in there somewhere –  Glen_b Feb 20 at 21:30
1  
@Glen_b: Why say likelihood's defined only up to an multiplicative constant? After all, when comparing non-nested models from different families of distribution (e.g. with AIC, or with the Cox test), you need to remember that constant. –  Scortchi Feb 20 at 21:43
    
@Scortchi the definition isn't mine! You'll have to take it up with R.A.Fisher. It's been that way from the start, I think (1921). That it's still defined that way, at least in the continuous case, see here, for example, at the sentence beginning 'More precisely,'. –  Glen_b Feb 20 at 21:52

2 Answers 2

Note that the help on the function logLik in R says that for lm models it includes 'all constants' ... so there will be a log(2*pi) in there somewhere, as well as another constant term for the exponent in the likelihood. Also, you can't forget to count the fact that $\sigma^2$ is a parameter.

$\cal L(\hat\mu,\hat\sigma)=(\frac{1}{\sqrt{2\pi s_n^2}})^n\exp({-\frac{1}{2}\sum_i (e_i^2/s_n^2)})$

$-2\log \cal{L} = n\log(2\pi)+n\log{s_n^2}+\sum_i (e_i^2/s_n^2)$

$= n[\log(2\pi)+\log{s_n^2}+1]$

$\text{AIC} = 2p -2\log \cal{L}$

but note that for a model with 1 independent variable, p=3 (the x-coefficient, the constant and $\sigma^2$)

Which means this is how you get their answer:

nrow(mtcars)*(log(2*pi)+1+log((sum(lm_mtcars$residuals^2)/nrow(mtcars))))
       +((length(lm_mtcars$coefficients)+1)*2)
share|improve this answer

The AIC function gives $2k -2 \log L$, where $L$ is the likelihood & $k$ is the number of estimated parameters (including the intercept, & the variance). You're using $n \log \frac{S_{\mathrm{r}}}{n} + 2(k-1)$, where $S_{\mathrm{r}}$ is the residual sum of squares, & $n$ is the sample size. These formulæ differ by an additive constant; so long as you're using the same formula & looking at differences in AIC between different models where the constants cancel, it doesn't matter.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.