This question pertains to Q Methodology, a research method used to study people's subjectivity.
Q embodies ontological and epistemological assumptions that sometimes differ markedly from mainstream ("R") / survey research.
I'm trying to move some of the discussion on this methodology on to CrossValidated, so please don't downvote this because the question seems weird :).
Also, I'd be great if someone could tag this with
I've run into an unexpected (and unwelcome) problem: contra Brown (1980: 279; 1971: 284) and Block (1961: 78) with my data the correlation coefficients for Pearson, Spearman and Kendall are not "virtually identical": the differences average to .026 and .07, ranging to a maximum difference of .17 and .19 (Spearman and Kendall vs Pearson, respectively). More troublingly, these differences in the underlying correlation matrix filter down to the "factor" extraction (PCA, for now): Factor loadings, (automatic) flaggings and factor zscores all change. As would be expected, the results are not completely different, but resulting factors are sufficiently different so as to require a substantively different factor interpretation.
If, as appears to be the case for my data (and maybe other's?), the choice of the correlation coefficient matters, I would like to have an ontologically/epistemologically grounded reason for choosing one of them (similar to the recent thoughtful discussion on this list concerning the ontological foundation of different extraction techniques raised by Rikki Dean and Peter Schmolck).
Is there a more extensive literature on this choice in the context of Q method that I am missing?
The few pieces I could find that do discuss the different correlation coefficients (mostly, Brown 1980, 1971 and Block 1961) seem to suggest Pearson, though some of the reasons (Block: easier computation, lack of familiarity) appear now obsolete.
I have also become unconvinced that Q-sorts (also under a forced, normal distribution, as is the case here) can be considered (as Pearson would require) interval data . It seems to me that we can either assume that the forced normal distribution is just a heuristic for measurement, or that respondents do, in fact, have bell-curved feelings about any given Q-set of items. In the latter case, interval data may be (very approximately) justified, in the former clearly not. It seems implausible to me that people would have the same response distribution to a Q-set including, say, one or two extremely disagreeable items (for some people, anyway), and another Q-set with "milder" item formulations. In the former example, the difference between, say -5 and -4, may very well be greater than the difference between -1 and 0.
Where does this leave us? Maybe at Spearman's or Kendall's, instead of Pearson's. That, of course, raises more questions (to which I have no answer yet):
- Is any given extraction method equally applicable to those rank-order coefficients? (Pearson's, conveniently, is a measure of linear association that would seem to be amenable to, say, PCA).
- How would the downstream results have to be interpreted differently for these coefficients (if at all)?
- Which to use: Kendall's or Spearman's?
Ps.: I have a preliminary hunch about how (at least Pearson's vs Spearman's) differ in a systematic way that bears upon the foundations of Q methodology. Upon cursory inspection of my data, Pearson's based analyses result in factors that are greatly anchored by agreement or disagreement in the extremes of the distribution. This seems to make sense, mathematically: Spearman's "spreads out" the distribution in the middle, somewhat counteracting the quadratic weighting in calculating the covariance. Brown (1980: 271) says such weighting of the extremes is sound, because those are the items about which people feel strongest, or most certain.
That may be so, but I worry that an inordinate weighting of similarities and differences on the extremes (as under Pearson's) may risk trivialising the factors. My resultant Pearson's factors are relatively more anchored by starkly formulated, divisive items (one about socialism, for example).
That per se, may not be a problem, but I worry that by merely wording an item in a more or less stark manner, the Q researcher might inadvertently "manufacture" factors anchored by such divisive items. I wonder, for instance, whether my "pro-socialism" item would have been so instrumental, had I worded it differently (say, something about democratic control of the economy). My worry is that Pearson's is overly influenced by these vagaries; Spearman's (at least) seems to draw relatively more attention towards similarities and differences towards the middle of the distribution. I wonder whether that might not be more in line with the holistic ambitions of Q. (This last part is still very preliminary thinking, and I hope to develop it into a broader-based paper soon).