I have a repeated measures design for a Likert scale with 10 items. This scale is assumed to be unidimensional and this is one of the research questions along with the scales measurement invariance (MI) over time.
I conduct the analysis as follows: First assessment is randomly split into two pieces, I apply PCA in the one half, CFA in the second half and then I wish to make MI comparisons between the second half of the first assessment and the second assessment.
PCA is supporting a one factor construct but not very strongly (λ = 2.7, Prop: 73%) while CFA is problematic to provide one factor structure (c2/df > 3, RMSEA = 0.12, GFI = 0.87).
Under that circumstances does it make sense to run and provide also the necessary MI comparisons between the two assessments?