0
$\begingroup$

I am aware that questions about factor analysis with mixed variable types have already been addressed. My situation is unique:

I have datasets from two samples that were administered the exact same 122 personality items. I wish to compare the EFA results for each sample, but the problem is that in one dataset all the items are 5-scale Likert types (strongly agree to strong disagree), and in the other dataset, 5 out of the 122 items happen to be dichotomous (yes/no).

The question at hand is whether I can do an EFA per dataset without excluding the 5 items that happen to be binary in one dataset. Whichever approach I take, it needs to be well-justified as the results will be published. Thus far, I have considered and attempted three approaches, which involve different correlation matrices:

Option 1: polychoric/tetrachoric correlation matrix

Under the strictest of interpretations, the items are all ordinal and categorical (polytomous/dichotomous), and in that sense I can obtain a matrix of polychoric and tetrachoric correlations for the odd dataset (for the other dataset, the matrix would be all polychoric).

I used Polymat-C (SPSS program) to obtain the matrix and run the EFA in SPSS, but was notified that the matrix was not positive definite. To make sure that the program was not at fault, I also obtained the matrix using R 'psych' function mixedCor, and ran the EFA in R, and again--not positive definite. I then attempted to exclude different sets of items to see if some were creating the problem but could not identify any as problematic.

Option 2: Pearson correlation matrix

Under a less strict interpretation, I could consider my items interval and continuous, as is done routinely in personality research, and use standard EFA procedures that rely on Pearson correlations. In this case, procedures can be run with no errors.

Option 3: Pearson, biserial and tetrachoric correlation matrix

Using the mixedCor function in 'psych' (R), I can obtain a matrix with: (1) Pearson correlations between the non-dichotomous items (2) biserial correlations between dichotomous and non-dichotomous (3) tetrachoric between dichotomous items. This would be used for the odd dataset, whereas an all-Pearson matrix would be used in the other.

As in option 2, the EFA procedures can be run with no errors. Looking at the results, the loadings of those 5 dichotomous items also appear to be more accurate* than when using strictly Pearson correlations (option 2).

*This requires further explanation: the 5 affected items are all part of a 30-item scale, so I am able to run a separate EFA or CFA on those 30 items to compare the results (i.e. item loadings) with the ones reported in existing publications. I can also compare the item loadings between the two datasets. I can tell that the loadings are unusually low when Option #2 is attempted compared to Option #3, but I admit that this is just "eyeballing" the results.

FINAL THOUGHTS

I am leaning towards Option 3, but I don't know if my circumstances warrant using a correlation matrix with different types of correlations, even if it only involves 5 odd items in one sample.

Looking for an informed opinion on which option is best, if any. Other options welcome but I am likely to stick to EFA (not PCA and other data reduction techniques, and confirmatory techniques are not appropriate as this is an exploratory analysis). I can also see excluding those 5 items altogether, but that is only a final resort.

Update: I have received feedback from other academics reassuring me that using a mixed correlation matrix for EFA (as noted in option 3) is not a problem. I realize this inquiry may not be helpful to others as it is too lengthy and case-specific. Should the moderators wish to close it, that would be fine.

$\endgroup$
  • $\begingroup$ Clarification: The 122 items are the same as in the stems are the same, but 5 have different response options, ¿correct? $\endgroup$ – Gregg H Mar 26 '18 at 23:56
  • $\begingroup$ Yes, that is correct. $\endgroup$ – AlexR Mar 27 '18 at 0:26
0
$\begingroup$

For option #1, the problem is most likely that the estimation method for the polychoric correlations is doing an item-by-item paired estimation. The more accurate approach would be to estimate all of the correlations simultaneously, thus resulting in a correlation matrix that was not semi-negative definite (assuming the protocol can converge on an acceptable solution).

Option #2 is indeed one approach that may give an approximation to a solution spanning fewer dimensions, though if I were reviewing, I would discourage publication. Other reviewers may be less strict.

Option #3, this is a reasonable method to obtain an approximate solution, though it still has concerns. The main issue is how you are determining that some items "appear to be more accurate" when you are conducting and exploratory data analysis. (If you have an idea of the structure, you should be conducting a confirmatory factor analysis.

The option that I would recommend would be categorical confirmatory factor analysis. This method can account for the different response options for different items (in that the tau-cuts for estimation would be fewer for those items).

Happy to provide more information if this seems like the option you'd like to pursue.

$\endgroup$
  • $\begingroup$ Since I am exploring the structure of a number of traits, confirmatory methods are not appropriate. Re your comments on #1: I believe the correlations were obtained simultaneously using default global=TRUE in 'mixedCor' function ('psych' R package), so not sure this is the issue. Re #3; sounds like you're saying it might be acceptable to use tetrachoric/biserial correlations for those 5 items rather than Pearson, but you're not convinced of my reasoning, so I added further context. See if it changes your opinion, as I'm very interested in finding proper justification for going with Option 3. $\endgroup$ – AlexR Mar 27 '18 at 1:26
  • $\begingroup$ I haven't used mixedCor package...so just confirm that global means there is a single multivariable probability distribution being estimated vs. global meaning the same tau-cuts are applied to all items; re #3, I'd encourage you to review Finney & DiStefano's chapter (Non-Normal and Categorical data in structural equation modeling) in Hancock & Mueller's SEM, A 2nd Course...this may provide a rationale to use that approach. However, I still have concern about the the EFA vs. CFA issue...with your clarification, it still sounds as though you are working with an established measure (CFA). $\endgroup$ – Gregg H Mar 27 '18 at 14:02
  • $\begingroup$ Global means that the global values of tau parameters are used as opposed to the pairwise values (as documentation states). I'm doing an exploratory factor analysis of several similar personality scales (not CFA). The structure of each scale is already known; their joint structure is not known. Looking back I should have compressed my inquiry to a single question: can I justify using a "mixed" correlation matrix for the dataset with 5 binary items (and has it been done before)? Looked at the Finney & DiStefano chapter, but SEM is not relevant to what I'm doing. Appreciate your thoughts $\endgroup$ – AlexR Mar 28 '18 at 2:39
  • $\begingroup$ 1. In this case, global is not producing multivariate distributions, but pairwise estimations...if you want to revisit option #1, it actually would be to your benefit to not use the global option, let the tau's be estimated freely from pair to pair, and then rerun the analysis. $\endgroup$ – Gregg H Mar 28 '18 at 13:54
  • $\begingroup$ 2. If the structure of each scale is known, and you want to show a different structure exists between two existing scales, then an EFA may be locating nothing more than random noise between potentially highly correlated items. I'm not saying you shouldn't do this, but at a minimum, you should still confirm (via CFA) that the original scales' measurement models are adequate (if not better than the final solution you decide upon). $\endgroup$ – Gregg H Mar 28 '18 at 13:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.