Item Response Theory vs Confirmatory Factor Analysis

Question

I was wondering what the core, meaningful differences are between Item Response Theory and Confirmatory Factor Analysis.

I understand that there are differences in the calculations (focusing more on item vs. covariances; log-linear vs. linear).

However, I have no idea what this means from a higher-level perspective - does this mean that IRT is better than CFA in some circumstances? Or for slightly different end-purposes?

Any musings would be useful as a scan of the research literature led to more a description of IRT and CFA than any useful comparison of the core differences between them.

Answer 1

@Philchalmers answer is on point, and if you want a reference from one of the leaders in the field, Muthen (creator of Mplus), here you go: (Edited to include direct quote)

An MPlus user asks: I am trying to describe and illustrate current similarities and differences between binary CFA and IRT for my thesis. The default estimation method in Mplus for categorical CFA is WLSMV. To run an IRT model, the example in your manual suggests to use MLR as the estimation method. When I use MLR, is the data input still the tetrachoric correlation matrix or is the original response data matrix used?

Bengt Muthen responds: I don't think there is a difference between CFA of categorical variables and IRT. It is sometimes claimed but I don't agree. Which estimator is typically used may differ, but that's not essential. MLR uses the raw data, not a sample tetrachoric correlation matrix. ... The ML(R) approach is the same as the "marginal ML (MML)" approach described in e.g. Bock's work. So using the raw data and integrating over the factors using numerical integration. MML being contrasted with "conditional ML" used e.g. with Rasch approaches.

Assuming normal factors, probit (normal ogive) item-factor relations, and conditional independence, the assumptions are the same for ML and for WLSMV, where the latter uses tetrachorics. This is because those assumptions correspond to assuming multivariate normal underlying continuous latent response variables behind the categorical outcomes. So WLSMV only uses 1st- and 2nd-order information, whereas ML goes all the way up to the highest order. The loss of info appears small, however. ML doesn't fit the model to these sample tetrachorics, so perhaps one can say that WLSMV marginalizes in a different way. It's a matter of estimator differences rather than model differences.

We have an IRT note on our web site:

http://www.statmodel.com/download/MplusIRT2.pdf

but again, the ML(R) approach is nothing different from what's used in IRT MML.

Source: http://www.statmodel.com/discussion/messages/9/10401.html?1347474605

Answer 2

In some ways you are right, CFA and IRT are cut from the same cloth. But it many ways they are quite different as well. CFA, or more appropriately item CFA, is an adaption of the structural equation/covariance modeling framework to account for a specific type of covariation between categorical items. IRT is more directly about modeling categorical variable relationships without using only first- and second-order information in the variables (it's full information, so its requirements generally aren't as strict).

Item CFA has several benefits in that it falls within the SEM framework, and therefore has very wide application to multivariate systems of relationships to other variables. IRT, on the other hand, primarily focuses on the test itself, though covariates can also be included in the test directly (e.g., see topics on explanatory IRT). I've also found that item modeling relationships are far more general in the IRT framework in that non-monotonic, non-parametric, or just plain customized item response models are easier to cope with because one doesn't have to worry about the sufficiency of using the polychoric correlation matrix.

Both frameworks have their pros and cons, but in general the CFA is more flexible when the level of modeling abstraction/inference is focused on the relationship within a system of variables, while IRT is generally preferred if the test itself (and items therein) are the focus of interest.

Answer 3

I believe Yves Rosseel discusses it briefly in slides 91-93 of his 2014 workshop: http://www.personality-project.org/r/tutorials/summerschool.14/rosseel_sem_cat.pdf

Taken from Rosseel (2014, link above):

Full information approach: marginal maximum likelihood

origins: IRT models (eg Bock & Lieberman, 1970) and GLMMs

...

the connection with IRT

• the theoretical relationship between SEM and IRT has been well documented:

Takane, Y., & De Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393-408.

Kamata, A., & Bauer, D. J. (2008). A note on the relation between factor analytic and item response theory models. Structural Equation Modeling, 15, 136-153.

Joreskog, K. G., & Moustaki, I. (2001). Factor analysis of ordinal variables: A comparison of three approaches. Multivariate Behavioral Research, 36, 347-387.

when are they equivalent?

• probit (normal-ogive) versus logit: both metrics are used in practice

• a single-factor CFA on binary items is equivalent to a 2-parameter IRT model (Birnbaum, 1968):

In CFA: ... In IRT: ... (see slide)

• a single-factor CFA on polychotomous (ordinal) items is equivalent to the graded response model (Samejima, 1969)

• there is no CFA equivalent for the 3-parameter model (with a guessing parameter)

• the Rasch model is equivalent to a single-factor CFA on binary items, but where all factor loadings are constrained to be equal (and the probit metric is converted to a logit metric)