I'm currently validating a french translation of a psychological questionnaire.
The original version is subdivided into two subscales. The 27 questions were answered on a Lickert scale (from 1 to 7).
254 participants completed it and performing a confirmatory factor analysis (with 2 factors) showed the following results:
Chisq (323) = 554.504, p < .001 ; RMSEA = 0.053 ; SRMR = 0.067 ; CFI = 0.816, GFI = 0.856 ; TLI = 0.8.
Cronbach's alphas are respectively: 0.74 and 0.82. I'm given to understand that the RMSEA and SRMR are okay but not the remaining measures of the fit.
But with that, I have absolutely no idea on how to improve it. So my questions are:
Should I change the questions and redo the survey? Or does a transformation/manipulation exist to have a better fit (while staying scientifically correct)?
In the frame of validation, is it ok to change the factorial structure to a three factors solution (I tried a PCA to see the structure and, even tho a 2 factors solution is ok, a 3-factor solution is way better)?
Thank you very much for your answers.
PS: I checked the normality of the data. None are normally distributed but two items seem to particularly deviate from the normal (most subjects who answered the same answer there). I do not know any other parameter I should check before performing the CFA.