Confirmatory factor analysis

Question

I'm currently validating a french translation of a psychological questionnaire.

The original version is subdivided into two subscales. The 27 questions were answered on a Lickert scale (from 1 to 7).

254 participants completed it and performing a confirmatory factor analysis (with 2 factors) showed the following results:

Chisq (323) = 554.504, p < .001 ; RMSEA = 0.053 ; SRMR = 0.067 ; CFI = 0.816, GFI = 0.856 ; TLI = 0.8.

Cronbach's alphas are respectively: 0.74 and 0.82. I'm given to understand that the RMSEA and SRMR are okay but not the remaining measures of the fit.

But with that, I have absolutely no idea on how to improve it. So my questions are:

Should I change the questions and redo the survey? Or does a transformation/manipulation exist to have a better fit (while staying scientifically correct)?

In the frame of validation, is it ok to change the factorial structure to a three factors solution (I tried a PCA to see the structure and, even tho a 2 factors solution is ok, a 3-factor solution is way better)?

Thank you very much for your answers.

PS: I checked the normality of the data. None are normally distributed but two items seem to particularly deviate from the normal (most subjects who answered the same answer there). I do not know any other parameter I should check before performing the CFA.

Answer 1

Should I change the questions and redo the survey? Or does a transformation/manipulation exist to have a better fit (while staying scientifically correct)?

• No. Usually, you would have two following courses of action. First, specify an alternative factor solution. In your case, for example, it might be a 3 or perhaps a 4-factor solution. Compare the fit among all factor solutions. Here you want to see whether alternative factor structures provide a noticeably better fit. Note that if alternative factor structures fit much better, you should be able to explain it theoretically too. This is a very common practice/strategy in survey validation (fit of alternative factor structures is reported too).

• Second, if you want to stick to a 2-factor solution specifically, but you want to improve its fit, you need to apply modification indices. Again this is a common way of improving model fit in a context of validating psychological measures. Here, the idea is to simply correlate residual variances of some items. Modification indices can be produced in both R or Mplus. Usually, you apply the modification with the highest index. Then rerun the model, and see if fit improved. If necessary, repeat the process by adding one modification at a time.

• Finally, I would like to point out the following. In current psychological survey validation practice or in psychometrics reporting GFI is not necessary as it has been shown to be a sub-optimal fit index. Also, Chi-square is only reported for historical reasons, as it is affected by both a) sample size and b) model complexity, as measured by the number of items/ indicators. Instead, it is common to use a combination of the following absolute fit indices (i.e. the RMSEA, and SRMR) and comparative fit indices(i.e. TLI and CFI). I think you will benefit a lot from reading the following post to get a better understanding of these indices.

• Also, note that while it is more robust if you have all of you fit indices showing good fit, in practice and in a lot of published research (e.g. see Campbell et al (2004) Entitlement scale with over 400 citations) you really end up with a combination of adequate and less than adequate fit indices. While this is not perfect, it is commonly the reality, and it is readily published, especially if you provide fit of alternative factor models.