3
$\begingroup$

I've been trying to find any references which have discussed the legitimacy of any item aggregation methods prior to conducting CFA, but I'm coming up short. Here's my context:

Large survey with 6 scenarios, 12 items per scenario, and roughly 4 items per scenario which map to 1 of 3 latent factors [see figure 1]. When I run CFA where latent factors map to each induvial item and add a method factor for each set of items within a scenario, I get marginally decent or bad fit statistics.

However, if I first sum the related items in each scenario [see figure 2], this new model produces really fantastic fit statistics. Is this type of aggregation ever allowed? If so, when is this allowed, and when is it not allowed? Why? And where are the references that talk about this?

Thanks for your help!

model structure

model with aggregated items within scenarios

$\endgroup$

2 Answers 2

8
$\begingroup$

Perhaps you can find something useful in the literature on (or related to) item parceling, for example:

Little, T. D., Cunningham, W. A., Shahar, G., & Widaman, K. F. (2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling, 9(2), 151-173.

Little, T. D., Rhemtulla, M., Gibson, K., & Schoemann, A. M. (2013). Why the items versus parcels controversy needn’t be one. Psychological Methods, 18(3), 285.

Little, T. D., Rioux, C., Odejimi, O. A., & Stickley, Z. L. (2022). Parceling in structural equation modeling: A comprehensive introduction for developmental scientists. Elements in Research Methods for Developmental Science.

Sterba, S. K., & Rights, J. D. (2023). Item parceling in SEM: A researcher degree‐of‐freedom ripe for opportunistic use. In R. H. Hoyle (Ed.), Handbook of Structural Equation Modeling (pp. 296-315). Guilford Press.

$\endgroup$
1
  • 3
    $\begingroup$ Thank you! It's always interesting how using the correct terminology can get you so much further :) $\endgroup$ Feb 16 at 22:45
6
$\begingroup$

Item Parceling

As @Christian Geiser suggests in his answer, you should look into the item parceling literature$^1$. Item parceling, according to Sterba (2011):

involves the averaging or summing of several raw items to form a single score, which can then be used as an indicator of a latent variable in a factor analysis model or structural equation model.

Hopefully, this answers your question:

Is this type of aggregation ever allowed?

as Sterba (2011) and the references provided by @Christian Geiser demonstrate that this (i.e., item parceling) is commonly done in practice.

Model fit

When I run CFA where latent factors map to each induvial item and add a method factor for each set of items within a scenario, I get marginally decent or bad fit statistics.

However, if I first sum the related items in each scenario [see figure 2], this new model produces really fantastic fit statistics. Is this type of aggregation ever allowed? If so, when is this allowed, and when is it not allowed? Why? And where are the references that talk about this?

Now, regarding your question about fit$^2$, I am not surprised that your second model (i.e., the one that used item parceling) fits better. My main reason for this is that it has been demonstrated that (paradoxically) many fit indices (and the RMSEA in particular) display worse fit as the number of items increases. See Shi et al. (2019) for more information on this point.

$^1$Also see Sterba (2019), which is my favorite article on this topic. Further, you can check out the Quantitude podcast episode on item parceling.

$^2$Also,there are additional considerations that should be made when evaluating the fit of a model that uses item parceling. For more information on this, see not only Sterba (2011), but also Sterba & MacCallum (2010) and Sterba & Rights (2017).

References

Shi, D., Lee, T., & Maydeu-Olivares, A. (2019). Understanding the model size effect on SEM fit indices. Educational and psychological measurement, 79(2), 310-334.

Sterba, S. K. (2011). Implications of parcel-allocation variability for comparing fit of item-solutions and parcel-solutions. Structural Equation Modeling: A Multidisciplinary Journal, 18(4), 554-577.

Sterba, S. K. (2019). Problems with rationales for parceling that fail to consider parcel-allocation variability. Multivariate behavioral research, 54(2), 264-287.

Sterba, S. K., & MacCallum, R. C. (2010). Variability in parameter estimates and model fit across repeated allocations of items to parcels. Multivariate Behavioral Research, 45(2), 322-358.

Sterba, S. K., & Rights, J. D. (2017). Effects of parceling on model selection: Parcel-allocation variability in model ranking. Psychological Methods, 22(1), 47.

$\endgroup$
2
  • 1
    $\begingroup$ This is very helpful. I appreciate all the references :) $\endgroup$ Feb 16 at 22:46
  • $\begingroup$ @Peter Wesley Odom great, glad it helped. $\endgroup$ Feb 16 at 22:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.