I need some advice to decide which aggregation method is better in my case. I want to create an aggregated variable which is a composite of 16 variables. All 16 variables are part of one big concept, so I think I could use PCA or CFA for this task. I need only one variable and not 16, since I will use it later as an independent variable in the regression analysis. My Cronbach's alpha is near 0.87 which is very good; however, since I have 16 items, it is not very reliable.
So first, I decided to use PCA, because I have a task of dimension reduction. However, a cumulative variance of the first factor is 0.55 and eigenvalues are low. I use predicted factor score in my regression and did not get the significant results. Then, I tried the
factor , pf command from
STATA and got much higher cumulative variance, 0.85, and eigenvalue for the first factor is above 8 (other methods like ML, PCF did not give me good results). I can also predict factor scores using this command. I am not sure, however, if I can use the command for EFA (
factor , pf as far as I understand is only for EFA, and for CFA I have to use SEM) to do CFA, and whether CFA is appropriate in this case.
My second question: are there some measures of goodness that could help me to choose between methods?
The last question is about $z$ scores: My 16 variables come from text mining and they are word counts that describe special topics, which could be aggregated to the bigger supertopic. Do I need to generate $z$ scores before doing PCA/CFA here? All variables are measured in the same way.