0
$\begingroup$

I ran a multiple regression using statsmodels. I wanted to verify my understanding of calculations for log-likelihood (ll), AIC and BIC. So I attempted to manually calculate the ll, AIC and BIC for the regression and compare my results to what I got from statsmodels. The results I got are a bit off for AIC and BIC.

Below is the statmodels output in an array format

import numpy as np

smdata = np.array([[-27362., -20881.],       #ll for y1 and y2
                   [5.473e+04, 4.177e+04],   #aic for y1 and y2
                   [5.477e+04, 4.181e+04]])  #bic for y1 and y2

Now, below is what I got from my attempt at manual estimation, the estimates for AIC and BIC are a bit off, not sure if they are due to rounding:

mdata = np.array([[-27362.332, -20880.994],   #ll for y1 and y2
                  [54734.664, 41771.988],     #aic for y1 and y2
                  [54771.464, 41808.788]])    #bic for y1 and y2

The formulae I used for the ll, AIC and BIC are below:

#ll
ll = -(n / 2) * np.log(2 * np.pi) - (n / 2) * np.log(rss / n) - n / 2

#aic
aic = -2 * ll + 2 * k

#bic
bic = -2 * ll + np.log(n) * k

Finally, the values for the estimations are below:

n = 11614,
k = 5
rss = np.array([75663.11462955, 24783.19428754])  #y1 and y2

I am satisified with the results for the ll. I just want to get some insights into why the AIC BIC values are a bit off.

$\endgroup$
2
  • 1
    $\begingroup$ The values are the same at print precision, 4 to 5 digits. To see a difference you need to print more decimals. Then the difference might be just numerical noise from the floating point computation (or other sources, e.g. convergence tolerance of parameter estimates) $\endgroup$
    – Josef
    Commented May 9, 2023 at 16:49
  • $\begingroup$ There are differences across packages for example in whether the scale estimate is included (in R) or not included (in statsmodels default) in the parameter count k. $\endgroup$
    – Josef
    Commented May 9, 2023 at 16:50

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.