I`m doing a confirmatory factor analysis (CFA) on responses to a 16 item questionnaire to compare models of PTSD. At least two pairs of items display substantive error correlations (/residual correlations). If I accommodate this, by specifying in the model that each of those pairs of items are allowed to correlated freely, I get virtually identical fit indices from three different models. However, without these modifications, I notice that models, which have fewer items in the clusters involving high error correlations, return better fit indices (i.e. these models relocate some of the items to another, highly correlated cluster). This is a consistent pattern. It would seem to me, then, that models are "rewarded" for "almost" isolating sets of items which display high error correlations. However, I can find no reference to such a tendency in the litterature. My question is, then, whether fit indices, such as CFI and TLI, can be inflated as a result of error correlations, and whether this inflation is more pronounced in smaller clusters?
Let me give an example. Here is one of the official PTSD clusters, typically referred to as the intrusion cluster:
- recurrent and intrusive distressing recollections of the event, including images, thoughts, or perceptions
- recurrent distressing dreams of the event
- acting or feeling as if the traumatic event were recurring
- intense psychological distress at exposure to internal or external cues that symbolize or resemble an aspect of the traumatic event
- physiological reactivity on exposure to internal or external cues that symbolize or resemble an aspect of the traumatic event
In my current analysis, items 1 and 3 display high error correlations. It could also be conceived as local dependence. Looking at the content, it's possible that symptom 1 acts as a prerequisite for symptom 3, i.e. one would typically have to recall something in order to act or feel that it is happening again.
Now, one of the models takes two items from another (highly correlated) cluster and adds them to the intrusion cluster. This model returns identical fit indices to the competing models, if the error correlation is not addressed. However, specifying item 1 and 3 to freely correlate in all models, the aforementioned model returns the best fit indices of all of the models. I'm thinking maybe the fit indices of those competing models are inflated, when the error correlation is not addressed. A similar thing happens in another cluster, which also feature substantive error correlations between two items.
I've come across no CFA study of PTSD, which points out indications of error correlations and adjusts for this before comparing models. This could be because no other study actually had this problem. I use a translated version of the questionnaire, and a poor translation of one or two items does seem to account for one of the occurences of error correlations. But still it seems odd to me, that IRT studies are so focused on potential violation of local dependence, while CFA studies in general appear to rarely mention such (potentiel) issues. So, again, does anyone know of potential impacts of substantive error correlations when evaluating fit of competing models, where one might find more information about this, or perhaps why so few CFA studies appear to focus on this potential problem?