Differences between tetrachoric and Pearson correlation

Question

Currently I'm analysing around 300 items in the field of education.

I'm interested in the dimensionality of the dataset. So I compute a matrix of tetrachoric correlation. The goal is to do a factor analysis on this matrix.

By curiosity I compare to a matrix of Pearson correlation, and the results are different. When I compute differences between the matrices I have slight differences : no null mean with min and max ranging from $-0.3$ to $0.3$.

Any idea? Usually you read that, for dichotomous item, values are closed.

Further, I have missing values because, due to very hard and very easy items, some modalities are never observed together. In this case do you have any good proposition (and reasons) to handle this problem? Suppress these items, Imputation, ...

Answer 1

My best bet is that you are facing large imbalance between your response categories, for some of your items.

If you assume that your binary responses reflect individual locations on an underlying latent (i.e., continuous) trait, then correlating the two variables is ok, provided the cutoff is close to the mean of the bivariate density, as shown below (here cutoffs were set symmetrically at $(.5,.5)$, for a correlation of 0.5):

In this case, Pearson correlation will underestimate the true linear relationship between the two latent traits, especially in the mid-range of the correlation metric. On the other hand, when the cutoffs are clearly asymmetrical on both continuous variables, the tetrachoric correlation will generally overestimate the true relationship. The following picture illustrates the ideal case.

library(polycor)
set.seed(101)
n <- 500
rho <- seq(0,1,length=500)
pc1 <- pc2 <- tc <- numeric(500)

for (i in 1:500) {
  data <- rmvnorm(n, c(0, 0), matrix(c(1, rho[i], rho[i], 1), 2, 2))
  x <- data[,1]; y <- data[,2]
  xb <- ifelse(x>=mean(x), 1, 0); yb <- ifelse(y>=mean(y), 1, 0)
  pc1[i] <- cor(x, y)
  pc2[i] <- cor(xb, yb)
  tc[i] <- polychor(xb, yb)
}

plot(pc1, pc2, cex=.6, col="red", xlab="True linear relationship",
     ylab="Observed correlation")
lines(lowess(pc1, pc2), col="red", lwd=2)
abline(0, 1, col="lightgray")
points(pc1, tc, cex=.6, col="blue")
lines(lowess(pc1, tc), col="blue", lwd=2)
legend("topleft", c("Pearson (0/1)","Tetrachoric"), col=c(2,4), lty=1, bty="n")

Now, you can play with the value of the cutoff, $\tau$, and see what happens when it is asymmetric and largely departs from the mean of the joint density of $x$ and $y$.

To complement @shabbychef's response, the phi coefficient is generally used with "truly" categorical variables (no hypothesis about a continuous generating process are made) and reduces to Pearson correlation in this case ($\sqrt{\chi^2}/n$). The problem is then to factor out a correlation matrix constructed in such a way because communalities become meaningless.

To avoid this problem, we may rely on parametric item response modeling, e.g. mixed-effects logistic model (in this case, no need to worry about the cutoff, since it is estimated), or non-parametric model, like Mokken scaling. In the latest case, we only assume monotonicity on the latent trait, but no functional form relating one's location on the latent trait and the outcome (i.e., the probability of endorsing the item). However, in your case, it would be a pain and would not allow you to identify a structure in your correlation matrix. But it may be used afterwards.

Finally, John Uebersax provides an in-depth discussion on the use of tetrachoric correlation in relation to latent trait modeling, see Introduction to the Tetrachoric and Polychoric Correlation Coefficients. Also, Nunnally discussed a long ago the advantages/disadvantages of relying on Pearson vs. Tetrachoric correlation coefficients in Factor Analysis, see e.g. pp. 570-573 (3rd ed.).

References

1. O'Connor, B. Cautions Regarding Item-Level Factor Analyses.
2. Bernstein, I.H., Teng, G. (1989). Factoring items and factoring scales are different: Spurious evidence for multidimensionality due to item categorization. Psychological Bulletin, 105, 467-477.
3. Edwards, J.H. and Edwards, A.W.F. (1984). Approximating the tetrachoric correlation coefficient. Biometrics, 40, 563.
4. Castellan, N.J. (1966). On the estimation of the tetrachoric correlation coefficient. Psychometrika, 31(1), 67-73.
5. Fitzgerald, P., Knuiman, M.W., Divitini, M.L., and Bartholomew, H.C. (1999). Effect of dichotomising a continuous variable on the assessment of familial aggregation: an empirical study using body mass index data from the Busselton Health Study. J. Epidemiol. Biostat., 4(4), 321-327.
6. Nunnally, J.C. and Bernstein, I.H. (1994). Psychometric Theory (Third ed.). McGraw-Hill.

Answer 2

Tetrachoric coefficient and Phi coefficient are indeed different. The tetrachoric coefficient is suitable for the following problem: Suppose there are two judges who judge cakes, say, on some continuous scale, then based on a fixed, perhaps unknown, cutoff, pronounce the cakes as "bad" or "good". Suppose the latent continuous metric of the two judges has correlation coefficient $\rho$. Now generate 300 cakes, have both judges taste each of them, and generate a 2x2 contingency table of "judge 1 bad/good" vs "judge 2 bad/good". Based on the data in this contingency table, the sample tetrachoric coefficient is an estimator (I believe it is the MLE, in fact), of the 'latent' correlation $\rho$. Note that the cutoffs employed by the two judges need not be known. The Phi coefficient views the pronouncements "bad", "good" themselves as the variable of interest, coded as 0/1, and is the sample Pearson coefficient of the 0/1 data. These are not the same.

edit in response to @pbneau's comments: my suspicion was that the tetrachoic and phi coefficients would diverge in the limit cases: as $\rho \to 0$ and as the cutoffs for the latent rating move away from the mean ratings. I tested this with my own code (in Matlab) for tetrachoric and phi coefficient. I tested with zero mean, unit variance Gaussian latent ratings with population correlations of 0.01 and 0.25, and with cutoffs of 0,0 and 1.5,-0.5. I ran 2048 experiments, each with 2048 'cakes'. The scatter fits for tetrachoric versus phi are shown here:

(looks like the image upload thing is not working; top row is $c1 = c2 = 0$, bottom is $c1 = 1.5, c2 = -0.5$, left column is $\rho = 0.01,$ right column is $\rho = 0.25$. The best fits along top row are $\rho^* = 1.5 \phi + 0$, $\rho^* = 1.4 \phi + 0.01$, along the bottom row, $\rho^* = 2.2 \phi$ and $\rho^* = 3.1 \phi - 0.04$. perhaps I can get this image hosted somewhere else that doesn't squash them so much...)

I'm not sure you can read the text on these (the preview looks bad); the upshot is that when the cutoffs are at the population mean, and thus the contingency tables are 'balanced' row-wise and col-wise (top row of plots), you get good correlation between the two metrics, but the tetrachoric tends to be a bit larger than phi. When the cutoffs are somewhat imbalanced, you get slightly worse correlation between the metrics, and the phi appears to 'shink' towards zero.

Thus my initial intuition was only half correct: the worst case appears to be as the latent $\rho$ moves away from zero, and the cutoffs move away from the latent means.

Answer 3

Well I think it has been widely adressed before as the PEARSON-YULE debate

The discrepancy between both measures seems to come from the fact that one assumes an underlying discrete random variable whereas for the other, the underlying (latent trait) is continuous. Apparently a bijective relationship between both exist according to Ekkstrom (2008). It's not an easy task for me to develop the whole think, Ekstrom paper does this in a clear way.

Ekstrom's paper http://statistics.ucla.edu/preprints/uclastat-preprint-2008:40