I am aware that questions about factor analysis with mixed variable types have already been addressed. My situation is unique:
I have datasets from two samples that were administered the exact same 122 personality items. I wish to compare the EFA results for each sample, but the problem is that in one dataset all the items are 5-scale Likert types (strongly agree to strong disagree), and in the other dataset, 5 out of the 122 items happen to be dichotomous (yes/no).
The question at hand is whether I can do an EFA per dataset without excluding the 5 items that happen to be binary in one dataset. Whichever approach I take, it needs to be well-justified as the results will be published. Thus far, I have considered and attempted three approaches, which involve different correlation matrices:
Option 1: polychoric/tetrachoric correlation matrix
Under the strictest of interpretations, the items are all ordinal and categorical (polytomous/dichotomous), and in that sense I can obtain a matrix of polychoric and tetrachoric correlations for the odd dataset (for the other dataset, the matrix would be all polychoric).
I used Polymat-C (SPSS program) to obtain the matrix and run the EFA in SPSS, but was notified that the matrix was not positive definite. To make sure that the program was not at fault, I also obtained the matrix using R 'psych' function mixedCor, and ran the EFA in R, and again--not positive definite. I then attempted to exclude different sets of items to see if some were creating the problem but could not identify any as problematic.
Option 2: Pearson correlation matrix
Under a less strict interpretation, I could consider my items interval and continuous, as is done routinely in personality research, and use standard EFA procedures that rely on Pearson correlations. In this case, procedures can be run with no errors.
Option 3: Pearson, biserial and tetrachoric correlation matrix
Using the mixedCor function in 'psych' (R), I can obtain a matrix with: (1) Pearson correlations between the non-dichotomous items (2) biserial correlations between dichotomous and non-dichotomous (3) tetrachoric between dichotomous items. This would be used for the odd dataset, whereas an all-Pearson matrix would be used in the other.
As in option 2, the EFA procedures can be run with no errors. Looking at the results, the loadings of those 5 dichotomous items also appear to be more accurate* than when using strictly Pearson correlations (option 2).
*This requires further explanation: the 5 affected items are all part of a 30-item scale, so I am able to run a separate EFA or CFA on those 30 items to compare the results (i.e. item loadings) with the ones reported in existing publications. I can also compare the item loadings between the two datasets. I can tell that the loadings are unusually low when Option #2 is attempted compared to Option #3, but I admit that this is just "eyeballing" the results.
FINAL THOUGHTS
I am leaning towards Option 3, but I don't know if my circumstances warrant using a correlation matrix with different types of correlations, even if it only involves 5 odd items in one sample.
Looking for an informed opinion on which option is best, if any. Other options welcome but I am likely to stick to EFA (not PCA and other data reduction techniques, and confirmatory techniques are not appropriate as this is an exploratory analysis). I can also see excluding those 5 items altogether, but that is only a final resort.
Update: I have received feedback from other academics reassuring me that using a mixed correlation matrix for EFA (as noted in option 3) is not a problem. I realize this inquiry may not be helpful to others as it is too lengthy and case-specific. Should the moderators wish to close it, that would be fine.
