0
$\begingroup$

I'm working with 5 variables related to "child and family health" using Item Response Theory to check the model and identify its parameters. My model is composed of items such as "Do you have a child with a learning or behavior problem?" and "Do you and your family members have health insurance or access to regular medical and dental care?" My model is well fitted: TLI 0.92 However, the coefficients that I've obtained showed that one item (cf2) has the difficulty at 31.375. I'm not sure if something is going under my radar and I'm wondering if someone could explain these results.

This is the simulated data, just to give you the chance to run my code.

df2 = structure(list(cf1 = c(0, 0, 0, 10, 0, 0, 0, 10, 10, 0, 10, 0, 
                                 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0), 
cf2 = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 10, 0, 10, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0), 
cf3 = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                             0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                             0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                             0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                             10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                             0, 0, 0, 0, 0, 0, 10, 0), cf4 = c(0, 10, 0, 0, 10, 0, 0, 10,                                                                                                                               10, 0, 10, 0, 10, 10, 0, 10, 0, 10, 0, 0, 0, 0, 0, 10, 10, 10,                                                                                                                               10, 10, 0, 0, 0, 0, 0, 10, 0, 0, 10, 0, 0, 0, 0, 0, 0, 10, 0,                                                                                                                               10, 0, 0, 0, 10, 10, 0, 10, 0, 0, 10, 10, 0, 10, 0, 0, 0, 0,                                                                                                                               0, 10, 0, 10, 10, 0, 10, 0, 10, 0, 10, 0, 0, 0, 0, 10, 10, 10,                                                                                                                               0, 10, 10, 10, 0, 0, 10, 10, 10, 0, 0, 0, 10, 0, 10, 0, 0, 10,                                                                                                                               0), 
cf5 = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 10, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                                                                           0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0,                                                                                                                                           0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                                                                           0, 0, 0, 0, 0, 0, 0, 10, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                                                                           0, 0, 0, 0)), 
row.names = c(NA, -100L), class = c("tbl_df", "tbl", "data.frame"))
mod_2pl_cf <- mirt(data = df2, model = 1, itemtype = "2PL"
    )
    coef(mod_2pl_cf, simplify=TRUE, 
         IRTpars= TRUE)
$\endgroup$
1
  • 2
    $\begingroup$ Can you explain your abbreviations, IRT etc, and as an edit to the post, not only as comments ... and please add relevant tags, the tag r should never be the only one! $\endgroup$ Mar 22 at 1:48

1 Answer 1

2
$\begingroup$

A couple of things. First, in an item response theory (IRT) context, a Tucker-Lewis Index (TLI) of 0.92 is not necessarily good. For example, Cai, Chung, and Lee (2023) found that adequate fit is obtained when the TLI is greater than 0.97-0.98. Second, and perhaps more importantly, the two items you provide do not look like typical items one would use to establish a psychometric scale using IRT (or other models for continuous latent variables, such as confirmatory factor analysis). Instead, they look more like covariates used in a structural equation model. I say this because it is important to remember the assumptions of IRT models. Perhaps the most important assumption is that of conditional independence, which is that an individual's item responses are independent of each other, conditional on their latent variable score $\theta$. It is hard for me to believe that this assumption can be reasonably made with the two example items you provide. For more information on IRT and its assumptions, see Bock & Gibbons (2021).

This is all to say that I am not surprised you obtained such an implausible difficulty value, as I do not believe IRT is the correct model for your application.

References

Bock, R. D., & Gibbons, R. D. (2021). Item response theory. John Wiley & Sons.

Cai, L., Chung, S. W., & Lee, T. (2023). Incremental model fit assessment in the case of categorical data: Tucker–lewis index for item response theory modeling. Prevention Science, 24(3), 455-466.

$\endgroup$
2
  • 1
    $\begingroup$ Thanks. I was thinking about your response today and it does make sense. I agree with you 100%. However, from a purely statistical framework, IRT models and CFA (with categorical estimators) are virtually the same. Strangely, the CFA model runs perfectly while the IRT gives this strange coef. Any thoughts on that ? (By the way, I'm just asking for academic purposes, my team and I decided to use network analysis with this data instead of latent-based models). Thanks $\endgroup$
    – Luis
    Mar 24 at 23:11
  • $\begingroup$ @Luis. Glad it helped! $\endgroup$ Mar 31 at 19:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.