9
$\begingroup$

I am exploring the psychometric properties of a 10-item self-report measure. I have about 400 cases in two independent samples. The items are completed on 4-point Likert scales. An EFA clearly supports a one-factor solution (e.g., first eigenvalue over 6, all others under 1) and Cronbach's alpha is good (e.g., .90). No item has low item-total correlation.

I originally wanted to do a CFA (EFA was just a follow-up after I saw CFA wasn't good) testing a one-factor model. To my surprise, fit for the model was relatively poor:

CFI=.91
TLI=.88
RMSEA=.13

Moreover, loadings for each of the items is quite good (.65+).

Oddly, the SRMR=.05, which is acceptable / good.

Modification indices suggest I correlate errors all over the place. If there was a clear rational to do so (e.g., some of the items have very similar wording) I would do this; however, all the measures are worded similarly, and correlating all the error terms would be odd and painful.

I have never seen a case like this. The measure is internally consistent and is clearly comprised of one factor in EFA, but it exhibits poor fit in the CFA. The results are congruent in both independent samples (from different continents). I tried a two-factor CFA (grouped 5 random items) and fit was the same, or even marginally better.

Here are my questions:

  1. Why is the fit according to CFI/TLI/RMSEA so poor given the EFA/Cronbach alpha/factor loadings?
  2. Why is the SRMR good while the other indices are not? I know they measure different things, but in my experience, they almost always converge.
  3. Should I correlate some of the errors?

Example items:

  • You have thoughts about your shortcomings
  • You have thoughts that are difficult to forget
  • You think about the situation all the time
$\endgroup$
9
$\begingroup$

That's pretty normal.

CFA is a much more stringent criterion than EFA. EFA attempts to describe your data, but CFA tests if the model is correct.

One reason for non-convergence is low average correlations (but then I'd expect RMSEA to be better). The chi-square test is essentially a test that your residuals are equal to zero, and RMSEA, TLI and CFI are transformations of the test.

Fit is always going to be better in a two factor solution than a one factor solution (they're nested).

Some more questions: What was your sample size? What's the average correlation? What's chi-square and df, what's the chi-square of the null model?

Should you add correlated errors? Perhaps, but when you do that you are introducing additional factors. With a fit like this you might need to add a lot, and then you end up with a mess - it's best if they are justified in some way. For example, your second and third items are about intrusive thoughts - that could be a justification.

$\endgroup$
  • 1
    $\begingroup$ Sample size is about 400 in each sample. What average correlation are you referring to? Chi-square in the model is 262.9, df = 35. $\endgroup$ – Behacad Apr 14 '13 at 18:35
  • $\begingroup$ Also, what is the alternative to the one-factor solution? The EFA suggests one factor, clearly, so it seems fishing for an alternative solution would be unusual. We only have the 10 items, so its not like we can add items. We could remove items, but all the loadings/correlations are strong! $\endgroup$ – Behacad Apr 14 '13 at 19:36
  • $\begingroup$ Average correlation is the average of the correlations in the matrix. If the correlations are all 0.3 this is different to if they're all 0.8 (say). If you're desperate for good fit, I'd remove items. Are you using Mplus? You could do esem if you are. $\endgroup$ – Jeremy Miles Apr 14 '13 at 20:07
  • $\begingroup$ I am using AMOS. $\endgroup$ – Behacad Apr 14 '13 at 23:06
  • $\begingroup$ Try a maximum likelihood extraction in SPSS - that should give you the same (or very similar) chi-square for a single factor. $\endgroup$ – Jeremy Miles Apr 15 '13 at 4:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.