2
$\begingroup$

I am running a CFA model (N=500) with a total of 5 factors: 3 of them include reverse items. I first run a reliability test psych::alpha(df_cronbach) which showed a good consistency in all factors.

Some context: 7 points Likert-scale; positive asymmetric distribution; est= Diagonally weighted least squares (DWLS); Lavaan

When running the CFA the model shows bad fit (SRMR, RMSEA, CFI).

The std. estimates of these reverse items are much lower than the other indicators within the same factors. This is happening for all of my factors that contain one or more reverse items.

For example, a factor that measures social norms is composed by 4 indicators (2 items are positively worded and 2 negatively worded - but recoded for the analysis). I inspected the residual correlation matrix resid(fit, "cor") and I saw a big shared error variance between the two reverse items (0.676). As expected, when I asked R to suggests me how to improve my model modindices(fit) the highest MI was related to the two reversed code items (139.671) item1_recode ~~ item2_recode

When allowing residuals to correlate between these 2 reverse items of the same factor the model fit improves, a lot.

To my knowledge, it should be a problem related to the fact that I am measuring the same construct but with a different method (type of wording), and this lead to a big unexplained variance. My problem is that I don't know best practises for this case.

Example of indicators:

A positive worded: I think that most people agree [...];

A negative worded: I think that most people disapprove [...] #reverse code

My questions:

Is there any best practice I should follow? Eliminate the indicators? Allowing residuals to correlate? My model fit get much better when allowing residuals to correlate between the two reverse items of the same factor (does it makes sense?). What about factors that contains only one reverse item, what I should do?

Thank you very much!

Fran

$\endgroup$

1 Answer 1

2
$\begingroup$

Allow residuals to correlate if you are fitting a model. Your two questions measure the same thing.

If you are constructing a scale, remove one.

You might also consider a method factor - that all negative questions load on, to account for people having differing response styles.

$\endgroup$
2
  • $\begingroup$ This CFA is my measurement model and I am going to test the structural model with a SEM. In the case of the social norm factor I have 4 indicators, 2 pos and 2 neg, not sure if you meant that the residuals correlations should be performed between the 2 neg only or ...? These 4 indicators ask the same thing using different words: "agree", "accept", "reject", "disapprove". Is there any specific reference that I could possibly use to justify this step? $\endgroup$
    – FLisa
    Commented May 20, 2020 at 12:25
  • $\begingroup$ There are lots of references out there, e.g. ncbi.nlm.nih.gov/pmc/articles/PMC4909292 $\endgroup$ Commented May 20, 2020 at 15:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.