In Q Methodology, it is common to logically flag (where FALSE sets the loading to 0) small or "insignificant" factor/component loadings before computing factor/component scores – prior to calculating a weighted average of raw scores (Watts & Stenner 2012: 128).
For context: Q is concerned with finding ideal-typical, shared, subjective viewpoints on some subject matter. To that end, participants rank-order a number of statements (40-80) on that subject. These ranks (roughly treated as interval-scaled, but that's another matter ...) are than factor analysed (or PCAed), with the data table turned sideways: participants are person-variables and statements are item-cases. Resultant factor scores are interpreted.
Under automatic flagging, loadings (of people-variables on factors/components) are retained for posterior calculations (of item-case-scores etc.) if:
- "the respondent correlates statistically significantly with that factor; the loadings of a respondent on a factor should exceed the multiplier for the statistical significance level (p = 0.05) divided by the square root of the number of statements [= statement-variables] [...]
- [...] the factor explains more than half of the common variance; that is, the square of the loading on that factor should exceed the sum of the squares of the factor loadings on the remaining factors" (van Exel et al. 2011: 388, footnote 8).
The second condition also takes care of what us Q people call "confounded Q-sorts" – that is, if a person-variable significantly loads on more than one factor, by only allowing one flag per loadings row.
To illustrate, here is an example of (un-flagged, raw) loadings and corresponding flags:
f1 f2 f3 Chris 0.77894318 0.09739471 -0.07221777 Johannes 0.31202968 0.70620248 0.22354582 Klas -0.02913268 0.81015347 -0.04978501 f1 f2 f3 Chris TRUE FALSE FALSE Johannes FALSE TRUE FALSE Klas FALSE TRUE FALSE
Obviously, "normal" (R-type, with person-cases and item-variables) exploratory factor analysts also compute factor scores, but I have been unable to find any reference to such a procedure (nothing in Thompson 2004, Child 2006:63 calls something vaguely "moderately rigorous").
I am skeptical about this procedure, which comes in addition to factor weighting/scoring.
The argument in favor of doing this, is, I think, not to pollute factor scores with noise (loosely, though not technically, error) from low-loading people-variables.
It seems to me
- superfluous, because different scoring mechanisms already weighs by (sometimes squared) loadings,
- dicey, because it might make interpretation/validity of downstream statistics way more complicated (correlations between factor scores, R2, etc.)
- somewhat dodgy, because it seems to "stack the deck" in favor of neater, cleaner, more distinct factor scores than the model really permits (effectively reducing the R2, without actually correcting its estimate).
Or, in question form:
- Is anyone outside of Q methodology doing something like this to compute factor scores?
- Am I right to worry that results will be "neater" than justified by the model?
Ps.: I should add, there are also Q proponents of manual flagging where above flags are set in a discretionary way by researchers, based on loadings, but also other information about person-variables and "what makes sense". That's a separate concern.
Pps.: I know this is close to a psychometric question, but please, bear with me – I am trying to get an outsider's (non-Q), mainstream statistics view on this.
Ppps.: if anyone could add a
qmethod tag to this question, that would be fantastic.
- van Exel J, de Graaf G, Rietveld P. (2011): "I can do perfectly well without a car!" -- An exploration of stated preferences for middle-distance travel. Transportation (38:383-407)
- Watts, Simon and Stenner, Paul (2012): Doing Q Methodological Research –Theory, Method and Interpretation. Thousand Oaks, CA: Sage
- Thompson, Bruce (2004): Exploratory and Confirmatory Factor Analysis. Washington, DC: APA.
- Child, Dennis (2006): The Essentials of Factor Analysis.