5
$\begingroup$

In reading various summaries on the similarities and differences between principal components and common factor models, I have noticed that there seems to be conflicting information about whether PCA models are reflective (e.g., a special case of traditional EFA models, as Fabrigar & Wegener's 2011 book seems to imply), formative (e.g., as this blog suggests), or neither (the impression I get from ttnphns's answer to this CV post, who describes components as linear functions of variables AND variables as linear functions of components), or both (as amoeba's comment on this thread suggests).

To be clear, I mean formative and reflective in the way described by LauraCS in the aforementioned blog, with formative meaning variation in the measured variables causing the variation in the principal component, in contrast to reflective meaning variation in the principal component causing variation in the measured variables

Can someone provide an accessible explanation of which is the case (reflective, formative, both, or neither) and why?

enter image description here

References

Fabrigar, L. F., & Wegener, D. T. (2011). Exploratory factor analysis. New York, NY: Oxford University Press.

$\endgroup$
11
  • 1
    $\begingroup$ Why neither? It's both. $\endgroup$
    – amoeba
    May 12, 2017 at 8:19
  • 2
    $\begingroup$ Please, explain in the question, in words, what's "formative" and "reflective" models, with commenting on the charts (defining the symbols there). $\endgroup$
    – ttnphns
    May 12, 2017 at 18:54
  • 1
    $\begingroup$ There have been a handful of answers here explaining conceptual differences between PCA and FA (classic versions of both). Among those, I will just indicate a pair of mine: 1, 2. Read other users' opinions, too. $\endgroup$
    – ttnphns
    May 12, 2017 at 20:09
  • 1
    $\begingroup$ (cont.) FA is a classic latent variable model: there is assumed latent factor(s) (which is modeled) that is the "cause" or "driver" of the manifest variables. Independent noise in each variable is the other source of their variability. Thus, the factor effects the variables, not back: "reflective" case. $\endgroup$
    – ttnphns
    May 12, 2017 at 20:09
  • 2
    $\begingroup$ jsakaluk, I'm not quite comfortable with both your terms because it isn't very clear if "causing" implied is mechanic or statistical. And I haven't read the blog source. So, I will refrain from labeling PCA this or that or neither. I hope that my point was clear enough for you to arrive at a decision how to label, if you need. PCA does not model causing/generative path; therefore you are in right to claim the direction of such path - if you need it - as you like. $\endgroup$
    – ttnphns
    May 12, 2017 at 20:20

1 Answer 1

3
$\begingroup$

[Disclaimer: Part of this has been published on my site back in 2011.]

PCA does not include an error model, which is one of the definitive characteristic of reflective measurement models: PCs are linear combinations of the observed variables, while in FA (and IRT) models each observed variable is modelled as a linear combination of factors (or latent variables), plus some variable-specific noise (unique factors). As such, the first PC is sometimes used as a weighted scale score, instead of the simple summated or totalled sum score of all item responses. As a rough approximation it is also used to select the number of relevant factors in exploratory FA (but other methods like ML, principal axis factoring, mean squares, etc. work as well or better in practice). So, at the very best, it may be seen as a formative measure, except it is not really a "psychometric measure", in my opinion. It has been shown that PCA can be rewritten with the observed variables as "dependent" variables, as in a reflective model (by Nunally and Bernstein in their Psychometric Theory textbook, IIRC), but still it doesn't account for measurement error, like is the case of formative models. So I guess it depends on whether you consider a measurement model or a regression framework (with causality assumptions).

Structural Equation (or Path) Models are probably the most representative models for a mix of these two approaches: a reflective measurement model is used to construct latent variables which in turn can be regressed onto other latent variables, which are themselves assumed to cause the variations of observed or manifest variables, as depicted in the following illustration.

enter image description here
(error terms and regression/path coefficients deliberately omitted)

With formative measures, the construct appeared as a manifestation of the items, such that variations at the level of the items induce variations in the construct (the reverse does not hold); items are not exchangeable and do not covary. In other words, these causal variables cannot be conditionally independent of each other with respect to the construct under investigation. With reflective measures, items are seen as manifestations of the hypothesized construct whose variations are directly reflected in items responses; items are exchangeable and any one can be safely removed without altering the construct. They appear to be aspects of the construct, or indicators of its magnitude.

The former obviously subsumes an idea of causality while the latter more or less resembles what we find in most unidimensional measurement scale in medical science. How we treat items will have consequence on the choice of an appropriate model (formative items are assumed to be measured without error, which is not the case for reflective items) but also on the interpretation of the results. This has led to various discussion in the clinimetrics and psychometrics literature, especially since many clinical variables can be conceived as both causal and indicator variable, or of intermediate type, which leaves us with this famous "psychometric embarrassment of riches" highlighted by Dennis Borsboom in The attack of the psychometricians:

Even within this concise set of questions to be answered we encounter no less than four radically different conceptualizations of the relation between conscientiousness and conscientious behaviors: a universe-sampling relation (generalizability theory), a formative causal relation (formative model), a reflective causal relation with the latent variable categorical (latent class model), and a reflective causal relation with the latent variable continuous (IRT). Moreover, as the IRT example shows, within each of these conceptualizations there are many more fine-grained choices to be made before we truly have a candidate model. Literally none of these choices are dictated by substantive theory. --- Dennis Borsboom (p. 435)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.