I am facing problem in getting the good fit indices. I have only 163 respondents with a total of 4 variables and 70 items (A: 5 components, 3-4 items each; B: 3 components, 8 items each; C: 4 components, 4-5 items each, D: 11 items). The initial values for fitness indices were:
ChiSq P-value = .000; RMSEA = .071; GFI = .586; SGFI = .558; CFI = .675; TLI = .663; NFI = .488; ChiSq/df = 1.815
Then, my supervisors suggested me to run EFA for the model but it ended up the items in the same group were being distributed to different groups, where the grouping of those items doesn't make sense at all as I adopted these constructs from past literature. The number of factors was not the same as the one proposed in past literature. Q1: Should I modify the model according to the EFA result?
Without modifying the model, I tried to do CFA for each construct separately and deleted items by referring to Modification Indices (MI). And I ran CFA for entire model again by deleting what I've deleted in the last step. However, the fitness indices are still poor. If I further delete items with highest MI, the result is even worse.
Then, I started over again by using Cronbach's Alpha to identified the items with lower reliability and deleted it. And I repeated the same steps by running CFA for individual constructs, deleting items according to MI, running CFA again for whole model, and deleting items according to MI. I've deleted 17 items from the model. The final result is as below:
ChiSq P-value = .000; RMSEA = .060; GFI = .690; SGFI = .660; CFI = .831; TLI = .822; NFI = .649; ChiSq/df = 1.587
Q2: May I know is there any other way that I could do to improve the fitness indices? If the problem is with my data, would I have to give up using SEM for analysis?