Factor analysis on a "rotating" subset of survey questions

Question

To help develop a survey instrument, I'd like to perform an exploratory factor analysis on a pool of 50-100 survey questions. (I'm open to discussing whether this is the best strategy for the project, but to keep the scope of this question reasonable, let's assume this is in fact a reasonable thing to do.) I expect to recover some four to eight factors.

Because of logistics of administering the surveys, I'd rather administer a randomly-selected subset of 20-30 questions instead of the whole thing. Using standard statistical software, is it still reasonable to do EFA on the kind of data this would produce? Or do I need to administer the entire survey to every respondent for it to work?

Answer 1

Because the design incorporates planned missingness, the data can be assumed to be missing completely at random and an imputation procedure could be adopted to deal with the missing data. I'd go with this option if you really have no idea how the items should load/number of factors to extract, since missingness tends to be quite high in planned missingness designs to just do something like listwise or pairwise deletion.

If you have some idea of the number of factors and where the items should load, another option is to use a more exploratory CFA. In this case, SEM adequately handles the missing data using full-information maximum likelihood estimation. If you choose this option, keep in mind that fit indices tend to show artificially better fit the more missingness you have. More info here.

Answer 2

EFA is based on the covariance between items. If items are well represented in the dataset, then you should be okay when estimating factor loadings and such. If you have many participants completing a large minority of random items, then your data should a good representation of the possible item combinations. This would thus look similar to if you only had participants complete all items. One issue is how completing only some of the items (or a specific subset) could impact response patterns, but I do not see this as a significant concern. A unorthodox method to be sure, but I cannot think of a salient critique (I stand to be corrected and down voted, which might occur shortly...).

This is basically like doing EFA with a ton of missing data, is it not? It still works, but you would need many more participants than usual since everyone is not completing everything.

Answer 3

This approach reminds me a lot of Synthetic Aperture Personality Assessment (SAPA). You may want to read further into it to see what you can learn about the method and judge how closely it resembles what you have in mind. If it's sufficiently equivalent for your purposes, the news appears to be good. This page says:

5) For some applications, data matrices are synthetically combined from sampling different items for different people. So called Synthetic Aperture Personality Assessement (SAPA) techniques allow the formation of large correlation or covariance matrices even though no one person has taken all of the items. To analyze such data sets, it is easy to form item composites based upon the covariance matrix of the items, rather than original data set. These matrices may then be analyzed using a number of functions (e.g., cluster.cor, factor.pa, ICLUST, principal , mat.regress, and factor2cluster.