Skip to main content
Tweeted twitter.com/StackStats/status/1060728835044532224
deleted 11 characters in body
Source Link
kjetil b halvorsen
  • 82.8k
  • 32
  • 201
  • 663

I am evaluating several logistic regression models predicting college student retention. I am using some basic and well-established predictors, such as high school GPA and SAT scores. I understand that when evaluating competing models, a lower AIC is generally preferable.

I had expected that usage of several years of data might improve my models, but I have noticed that the more years of data records that I add to the model, the more the AIC increases. For instance, here are the AICs for several models built using more or less years of student records (going back in time, relative to the most recent academic year):

  • Using 9 years of data: AIC = 4314
  • Using 5 years of data: AIC = 2789
  • Using 4 years of data: AIC = 2312
  • Using 3 years of data: AIC = 1776
  • Using 2 years of data: AIC = 1139
  • Using 1 year of data: AIC = 512

Do these results mean that the AIC becomes inflated or less meaningful with larger sample sizes, or does it imply that each year of records in my dataset are so dramatically different from one another that I probably need to fit a different model for each separate year, or something else?

Thanks!!

I am evaluating several logistic regression models predicting college student retention. I am using some basic and well-established predictors, such as high school GPA and SAT scores. I understand that when evaluating competing models, a lower AIC is generally preferable.

I had expected that usage of several years of data might improve my models, but I have noticed that the more years of data records that I add to the model, the more the AIC increases. For instance, here are the AICs for several models built using more or less years of student records (going back in time, relative to the most recent academic year):

  • Using 9 years of data: AIC = 4314
  • Using 5 years of data: AIC = 2789
  • Using 4 years of data: AIC = 2312
  • Using 3 years of data: AIC = 1776
  • Using 2 years of data: AIC = 1139
  • Using 1 year of data: AIC = 512

Do these results mean that the AIC becomes inflated or less meaningful with larger sample sizes, or does it imply that each year of records in my dataset are so dramatically different from one another that I probably need to fit a different model for each separate year, or something else?

Thanks!!

I am evaluating several logistic regression models predicting college student retention. I am using some basic and well-established predictors, such as high school GPA and SAT scores. I understand that when evaluating competing models, a lower AIC is generally preferable.

I had expected that usage of several years of data might improve my models, but I have noticed that the more years of data records that I add to the model, the more the AIC increases. For instance, here are the AICs for several models built using more or less years of student records (going back in time, relative to the most recent academic year):

  • Using 9 years of data: AIC = 4314
  • Using 5 years of data: AIC = 2789
  • Using 4 years of data: AIC = 2312
  • Using 3 years of data: AIC = 1776
  • Using 2 years of data: AIC = 1139
  • Using 1 year of data: AIC = 512

Do these results mean that the AIC becomes inflated or less meaningful with larger sample sizes, or does it imply that each year of records in my dataset are so dramatically different from one another that I probably need to fit a different model for each separate year, or something else?

More attention for more tags; tags added
Link
Carl
  • 13.3k
  • 7
  • 55
  • 115
Source Link
lewispgj
  • 53
  • 1
  • 3

How is the Akaike information criterion (AIC) affected by sample size?

I am evaluating several logistic regression models predicting college student retention. I am using some basic and well-established predictors, such as high school GPA and SAT scores. I understand that when evaluating competing models, a lower AIC is generally preferable.

I had expected that usage of several years of data might improve my models, but I have noticed that the more years of data records that I add to the model, the more the AIC increases. For instance, here are the AICs for several models built using more or less years of student records (going back in time, relative to the most recent academic year):

  • Using 9 years of data: AIC = 4314
  • Using 5 years of data: AIC = 2789
  • Using 4 years of data: AIC = 2312
  • Using 3 years of data: AIC = 1776
  • Using 2 years of data: AIC = 1139
  • Using 1 year of data: AIC = 512

Do these results mean that the AIC becomes inflated or less meaningful with larger sample sizes, or does it imply that each year of records in my dataset are so dramatically different from one another that I probably need to fit a different model for each separate year, or something else?

Thanks!!