Where can I find information about using SPSS for EFA and CFA? Is PCA (two samples) and reliability sufficient for scale development? Context:
I am in the process of developing a scale for my thesis.  My advisor has guided me to using SPSS PCA to complete my analyses.  Initially we reduced my scale to 3 factors (her insistence), and we collected a second sample with the 3 factor scale.  I have had issues with the 3 factors loading appropriately.
Participants respond differently to the third factor based on age and gender, and the loadings are coming in at .45 and then .42 in another factor.  When I consider the two factor model (15 item scale vs. 10 item scale), all loading is .65 or higher, and responding is not effected by gender or age.
Questions:


*

*What do you suggest I read in order to learn more about using SPSS PCA for scale development?

*Should I use SEM?

*Are there additional analyses I can conduct on this scale to determine the fit/factors?

 A: I do not recommend (see here towards the end) you to use PCA for scale (construct) development/validation. Only Factor Analysis in a true sense (FA) might be used in such a serious task. PCA is a technique to summarize, with small number of latents, the shape of multivariate data ellipsoid, that is, it aims to restore euclidean distances between data points with those latents. FA is a technique to model, with small number of latents, the associations (correlations/covariances) between items (variables). PCA is weak in such task because it unreallistically assumes that associations are produced by all of variability, while in fact only part of variability - called communality - is responsible for associations; the other part being unique variability of individual items.
As for "should I use SEM" question, I could add that Confirmatory Factor Analysis (CFA) is very useful in construct development. Particularly, in comparing factor structures. More easy still good alternative to CFA in such comparing is Procrustes Analysis.
A: Resources on EFA for scale construction


*

*I have a post listing several academic articles that provide guidance on using EFA to the task of scale construction.


Is PCA and reliability enough for scale development


*

*Scale development done well is a complex task. It generally involves a wide range of techniques. While exploratory factor analysis (EFA), confirmatory factor analysis (CFA), PCA, and reliability analysis, are relevant, there are other important techniques, such as item generation strategies, expert sorting techniques, and various strategies for seeing the relationship between scale scores and other variables.

*I prefer EFA to PCA because EFA assumes latent factors (see my previous discussion of the differences). However, both techniques are exploratory, and thus can be contrasted with CFA. 

*In the early and intermediate stages of scale development, I think EFA is generally more useful than CFA. It's good to let the data speak to you. Cross-loadings, or items that load maximally on the "wrong" factor will typically be more evident in EFA. 

*However, CFA can be applied in both exploratory and confirmatory ways. Thus, in skilled hands, CFA could still be useful to explore the factor structure. I.e., exploratory approaches to CFA make extensive use of modification indices and explore many different possible factor structures.

*In later stages of scale development, CFA can be more useful. Once the scale is almost finalised, you can check the fit of the proposed scale structure. You can check invariance of the factor structure across groups or over time. You can see whether assuming tau-equivalence is appropriate. CFA is also particularly useful if you want to compare the fits of multiple models of the factor structure of a set of items .


Comments on SPSS


*

*SPSS provides tools for EFA. see here for some links to tutorials

*If you want to do CFA, then SPSS has Amos. Of course, there are a wide range of tools for doing SEM.

