Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Currently I'm analysing around 300 items in the field of education.

I'm interested in the dimensionality of the dataset. So I compute a matrix of tetrachoric correlation. The goal is to do a factor analysis on this matrix.

By curiosity I compare to a matrix of Pearson correlation, and the results are different. When I compute differences between the matrices I have slight differences : no null mean with min and max ranging from $-0.3$ to $0.3$.

Any idea? Usually you read that, for dichotomous item, values are closed.

Further, I have missing values because, due to very hard and very easy items, some modalities are never observed together. In this case do you have any good proposition (and reasons) to handle this problem? Suppress these items, Imputation, ...

share|improve this question
How many individuals are used to estimate the item intercorrelations? What software is used for computing both correlations? –  chl Sep 26 '10 at 15:42
matrix is 5000 x 300. Software is SAS. PROC CORR and PLCORR. Convergence was not reached so the number of iterations have been changed to 100 iterations. –  pbneau Sep 26 '10 at 17:29

3 Answers 3

up vote 8 down vote accepted

My best bet is that you are facing large imbalance between your response categories, for some of your items.

If you assume that your binary responses reflect individual locations on an underlying latent (i.e., continuous) trait, then correlating the two variables is ok, provided the cutoff is close to the mean of the bivariate density, as shown below (here cutoffs were set symmetrically at $(.5,.5)$, for a correlation of 0.5):

alt text

In this case, Pearson correlation will underestimate the true linear relationship between the two latent traits, especially in the mid-range of the correlation metric. On the other hand, when the cutoffs are clearly asymmetrical on both continuous variables, the tetrachoric correlation will generally overestimate the true relationship. The following picture illustrates the ideal case.

n <- 500
rho <- seq(0,1,length=500)
pc1 <- pc2 <- tc <- numeric(500)

for (i in 1:500) {
  data <- rmvnorm(n, c(0, 0), matrix(c(1, rho[i], rho[i], 1), 2, 2))
  x <- data[,1]; y <- data[,2]
  xb <- ifelse(x>=mean(x), 1, 0); yb <- ifelse(y>=mean(y), 1, 0)
  pc1[i] <- cor(x, y)
  pc2[i] <- cor(xb, yb)
  tc[i] <- polychor(xb, yb)

plot(pc1, pc2, cex=.6, col="red", xlab="True linear relationship",
     ylab="Observed correlation")
lines(lowess(pc1, pc2), col="red", lwd=2)
abline(0, 1, col="lightgray")
points(pc1, tc, cex=.6, col="blue")
lines(lowess(pc1, tc), col="blue", lwd=2)
legend("topleft", c("Pearson (0/1)","Tetrachoric"), col=c(2,4), lty=1, bty="n")

alt text

Now, you can play with the value of the cutoff, $\tau$, and see what happens when it is asymmetric and largely departs from the mean of the joint density of $x$ and $y$.

To complement @shabbychef's response, the phi coefficient is generally used with "truly" categorical variables (no hypothesis about a continuous generating process are made) and reduces to Pearson correlation in this case ($\sqrt{\chi^2}/n$). The problem is then to factor out a correlation matrix constructed in such a way because communalities become meaningless.

To avoid this problem, we may rely on parametric item response modeling, e.g. mixed-effects logistic model (in this case, no need to worry about the cutoff, since it is estimated), or non-parametric model, like Mokken scaling. In the latest case, we only assume monotonicity on the latent trait, but no functional form relating one's location on the latent trait and the outcome (i.e., the probability of endorsing the item). However, in your case, it would be a pain and would not allow you to identify a structure in your correlation matrix. But it may be used afterwards.

Finally, John Uebersax provides an in-depth discussion on the use of tetrachoric correlation in relation to latent trait modeling, see Introduction to the Tetrachoric and Polychoric Correlation Coefficients. Also, Nunnally discussed a long ago the advantages/disadvantages of relying on Pearson vs. Tetrachoric correlation coefficients in Factor Analysis, see e.g. pp. 570-573 (3rd ed.).


  1. O'Connor, B. Cautions Regarding Item-Level Factor Analyses.
  2. Bernstein, I.H., Teng, G. (1989). Factoring items and factoring scales are different: Spurious evidence for multidimensionality due to item categorization. Psychological Bulletin, 105, 467-477.
  3. Edwards, J.H. and Edwards, A.W.F. (1984). Approximating the tetrachoric correlation coefficient. Biometrics, 40, 563.
  4. Castellan, N.J. (1966). On the estimation of the tetrachoric correlation coefficient. Psychometrika, 31(1), 67-73.
  5. Fitzgerald, P., Knuiman, M.W., Divitini, M.L., and Bartholomew, H.C. (1999). Effect of dichotomising a continuous variable on the assessment of familial aggregation: an empirical study using body mass index data from the Busselton Health Study. J. Epidemiol. Biostat., 4(4), 321-327.
  6. Nunnally, J.C. and Bernstein, I.H. (1994). Psychometric Theory (Third ed.). McGraw-Hill.
share|improve this answer

Tetrachoric coefficient and Phi coefficient are indeed different. The tetrachoric coefficient is suitable for the following problem: Suppose there are two judges who judge cakes, say, on some continuous scale, then based on a fixed, perhaps unknown, cutoff, pronounce the cakes as "bad" or "good". Suppose the latent continuous metric of the two judges has correlation coefficient $\rho$. Now generate 300 cakes, have both judges taste each of them, and generate a 2x2 contingency table of "judge 1 bad/good" vs "judge 2 bad/good". Based on the data in this contingency table, the sample tetrachoric coefficient is an estimator (I believe it is the MLE, in fact), of the 'latent' correlation $\rho$. Note that the cutoffs employed by the two judges need not be known. The Phi coefficient views the pronouncements "bad", "good" themselves as the variable of interest, coded as 0/1, and is the sample Pearson coefficient of the 0/1 data. These are not the same.

edit in response to @pbneau's comments: my suspicion was that the tetrachoic and phi coefficients would diverge in the limit cases: as $\rho \to 0$ and as the cutoffs for the latent rating move away from the mean ratings. I tested this with my own code (in Matlab) for tetrachoric and phi coefficient. I tested with zero mean, unit variance Gaussian latent ratings with population correlations of 0.01 and 0.25, and with cutoffs of 0,0 and 1.5,-0.5. I ran 2048 experiments, each with 2048 'cakes'. The scatter fits for tetrachoric versus phi are shown here: bunch o scatter fits

(looks like the image upload thing is not working; top row is $c1 = c2 = 0$, bottom is $c1 = 1.5, c2 = -0.5$, left column is $\rho = 0.01,$ right column is $\rho = 0.25$. The best fits along top row are $\rho^* = 1.5 \phi + 0$, $\rho^* = 1.4 \phi + 0.01$, along the bottom row, $\rho^* = 2.2 \phi$ and $\rho^* = 3.1 \phi - 0.04$. perhaps I can get this image hosted somewhere else that doesn't squash them so much...)

I'm not sure you can read the text on these (the preview looks bad); the upshot is that when the cutoffs are at the population mean, and thus the contingency tables are 'balanced' row-wise and col-wise (top row of plots), you get good correlation between the two metrics, but the tetrachoric tends to be a bit larger than phi. When the cutoffs are somewhat imbalanced, you get slightly worse correlation between the metrics, and the phi appears to 'shink' towards zero.

Thus my initial intuition was only half correct: the worst case appears to be as the latent $\rho$ moves away from zero, and the cutoffs move away from the latent means.

share|improve this answer
@shabbychef Yes, it's the MLE of correlation in a two-way table, see Hamdam (1970), j.mp/9SN7Lk, or Brown & Benedetti (1977), j.mp/aJjjzu. –  chl Sep 26 '10 at 17:11
@chl thanks, I thought I that was the case, but it always seems like a too remarkable fact because the cutoffs are unknown. –  shabbychef Sep 26 '10 at 20:31
@shabbychef Yes, the threshold remains unknown. To my opinion, John Uebersax provides a great overview on TCs correlation, j.mp/bnhem7. –  chl Sep 26 '10 at 20:34
Thanks for explanation and references. But what i would know is when values obtained from tetrachoric are slightly different of person coefficient. Because they're supposed to be close. Indeed, in many programs, for estimation, Pearson r are used as start values for algorithm which compute tetrachoric correlation. –  pbneau Sep 27 '10 at 7:33
@chl: not a problem. I like that you posted R code (I am slowly learning R); my version is in homebrew Matlab, which I cannot share. Having code that the OP can test is definitely the way to go. –  shabbychef Sep 28 '10 at 16:40

Well I think it has been widely adressed before as the PEARSON-YULE debate

The discrepancy between both measures seems to come from the fact that one assumes an underlying discrete random variable whereas for the other, the underlying (latent trait) is continuous. Apparently a bijective relationship between both exist according to Ekkstrom (2008). It's not an easy task for me to develop the whole think, Ekstrom paper does this in a clear way.

Ekstrom's paper http://statistics.ucla.edu/preprints/uclastat-preprint-2008:40

share|improve this answer
do you care to expand upon this comment? –  Macro Jun 27 '12 at 12:18
Thanks for the follow-up, and welcome to this site! –  chl Jun 27 '12 at 19:20

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.