1
$\begingroup$

I'm working with 4 nested models using ordinal regression (same sample, n=344, and dependent variable across models). The -2LL for each successive model increases and becomes statistically significant. However, the AICc and BIC numbers also rise quite a bit. I know that the -2LL shows how well the models fit the data compared to a baseline intercept-only model. I also know that it's standard practice to choose the model with the lowest AICc or BIC. However, my conflicting results seem to indicate that the full model is the worst fitting model according to AICc/BIC and the best fitting model according to -2LL. Here are the results (for simplicity, I'm only reporting AICc):

Model 1: four categorical independent variables, -2LL=14.37, AICc=365.33 Model 2: adds a fifth categorical independent variable, -2LL=26.74*, AICc=532.51 Model 3: adds a sixth categorical independent variable, -2LL=37.57*, AICc=591.58 Model 4: adds a scaled continuous variable, -2LL=40.28*, AICc-=727.461

I'd appreciate any help in how to interpret the model fit given these conflicting results. Also, if you have a statistics book or article that helps explain your answer, that would be great. Thanks!

$\endgroup$

1 Answer 1

1
$\begingroup$

As you enter more parameters to your model, it will tend to be closer to the data. AIC and BIC penalize for the number of parameter to prevent overfitting. So, you may want to use AIC and BIC to pick the best fitting model and then assess the model fit using criteria like likelihood etc.

$\endgroup$
1
  • $\begingroup$ Yes, and I think AIC is more appropriate. Assessing 4 models is pushing it a bit model uncertainty-wise, as having 4 opportunities distorts final standard errors and uncertainty intervals, plus p-values if you like those by mistake. $\endgroup$ Apr 4 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.