I have a data relating to a questionnaire that takes form of a knowledge test, in which each question has multiple correct responses and the respondents can check all responses they consider to be correct to a particular question. That is, the questions are "multiple response" type, to select 1+ response in each question. I wanted to perform a factor analysis on the data to psychometrically validate the questionnaire; however, I have never done a psychometric analysis on a questionnaire with multiple "positive" responses selected. Is this even possible to perform factor analysis on such data and if not is there any alternative analysis that lends itself to such data?
$\begingroup$
$\endgroup$
3
-
$\begingroup$ A multiple response question is a set of binary variables (1 = selected vs 0 = not selected). So, each item is seen as if a separate question with two responses. You could do factor analysis on a totality of such variables. But the problem is that classic FA is usally considered appropriate for interval scale data. One could treat Likert rating scale as interval, but not dichotomous one as interval. The topic "Factor analysis for binary data" has been several times discussed on this site. $\endgroup$– ttnphnsApr 3, 2022 at 23:45
-
$\begingroup$ start from stats.stackexchange.com/q/215404/3277 $\endgroup$– ttnphnsApr 3, 2022 at 23:48
-
$\begingroup$ Maybe also: stats.stackexchange.com/questions/18617/… $\endgroup$– kjetil b halvorsen ♦Nov 15, 2022 at 18:15
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
As @BulkySplash says, factor analysis is typically not applied to items coded as correct or incorrect (i.e., as dichotomous, or 0/1). Though it is possible. One option would be to simply treat measured variables as continuous, though this can be problematic (e.g., see Robitzsch, 2020; and Robitzsch, 2022 for more information). Another option would be to estimate factor analysis parameters using tetrachoric correlations, which can be easily done in all modern factor analysis/structural equation modeling (SEM) software packages (e.g., lavaan, mplus).
You could also try to use item response theory (IRT) methods such as the two-parameter logistic model (2PL) or three-parameter logistic model (3PL; which accommodates the presence of guessing). For more information on IRT methods, and their relationship with factor analysis see Cai (2013), Wirth & Edwards (2007), and Chen & Zhang (2021).
References
Cai, L. (2013). Factor analysis of tests and items.
Chen, Y., & Zhang, S. (2021). Estimation methods for item factor analysis: An overview. Modern Statistical Methods for Health Research, 329-350.
Robitzsch, A. (2020, October). Why ordinal variables can (almost) always be treated as continuous variables: Clarifying assumptions of robust continuous and ordinal factor analysis estimation methods. In Frontiers in education (Vol. 5, p. 589965). Frontiers Media SA.
Robitzsch, A. (2022). On the Bias in Confirmatory Factor Analysis When Treating Discrete Variables as Ordinal Instead of Continuous. Axioms, 11(4), 162.
Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: current approaches and future directions. Psychological methods, 12(1), 58.
$\begingroup$
$\endgroup$
Factor analysis is a model to measure latent variables. Usually it is applied to questionnaire items with no right or wrong, and instead used to identify items which correlate and load onto the same factor. Your questionnaire seems to be a test, so I would look into classical test theory instead. You can start by building sum scores, e.g. by counting the amount of correct responses.
