2
$\begingroup$

I'm writing a CFA paper, and I have run into some trouble interpreting the AIC and BIC. This is my first paper using continuous variables, thus the first time I will be reporting these fit statistics and I'm still learning the SEM method overall so bear with me, please.

The paper in question looks at an existing psychometric, but I am scoring it in a slightly different way than was intended. The measure has a frequency of experience and a distress element in it, and usually the frequency of experience is analysed for best fitting model with the distress element later correlated with the model. I have summed both of these elements (frequency scores + distress scores) to produce a new single set of scores which I have then run within a CFA framework (5 factor model using MLR).

The other fit indices look great, however, the AIC and BIC look like this:

CFA on the different elements of the measure:

Frequency : AIC= 12313.226 BiC: 12602.260
Distress  : AIC= 10318.698 BIC: 10607.731
Summed    : AIC= 22039.130 BIC: 22328.163

How would I go about interpreting these values?

$\endgroup$
12
  • 1
    $\begingroup$ Can you please clarify exactly what your question is? You seem to be indicating there is something wrong with your AIC/BIC values but I'm not sure what you're referring to. $\endgroup$
    – Macro
    Apr 20 '12 at 14:43
  • 1
    $\begingroup$ Are you referring to the magnitude of the absolute values? ICs should not be interpreted on an absolute scale. All seems to be in order $\endgroup$
    – Momo
    Apr 20 '12 at 14:58
  • $\begingroup$ I was talking with a collegue today and showed him the results, he mentioned that the lowest AIC and BIC was a preferable model and by summing the Freq+Distress elements of the measure may have done something as the AIC and BIC looks quite large. To clarify it is the Summed scores I wish to use the AIC and BIC fit statistics look very large when compared to the freq and distress. $\endgroup$
    – Rave
    Apr 20 '12 at 15:10
  • $\begingroup$ As Momo said, the magnitude of an AIC/BIC value should not be interpreted on its own - only AIC/BIC values relative to each other. $\endgroup$
    – Macro
    Apr 20 '12 at 17:05
  • 2
    $\begingroup$ I'm not sure I follow all of this but the summed and non-summed AICs are on different scales (since the likelihood scales with the sample size, which is different in the summed and non-summed groups), so you can't compare them directly. You can only compare them when the data set remains fixed (so the likelihoods are on the same scale). $\endgroup$
    – Macro
    Apr 20 '12 at 22:51
2
$\begingroup$

You can check out this book: "Model Selection and Multimodel Inference A Practical Information-Theoretic Approach", Burnham & Anderson 2nd. Ed.

$\endgroup$
2
  • $\begingroup$ Welcome to the site, @user13138. Can you say more about this book & why it will address the OP's question? $\endgroup$ Aug 16 '12 at 3:10
  • 3
    $\begingroup$ As Macro said, Section 2.11.1 explains that when the data sets are different, you cannot compare the AIC's. I think that is the reason why the criteria for the summed model are significantly different. Gung, don’t you think the one who asked the question should do a little bit of research him/herself? $\endgroup$
    – Stat
    Aug 16 '12 at 3:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.