# Variable ordering using PCA

I've downloaded a script to draw a correlation matrix using colored circles. This script allows to order variables using PCA, but I'm not sure how it works. The code responsible for ordering is below:

if (order) {
if(!n==m){
stop("The matrix must be squre if order is TRUE!")
}
x.eigen <- eigen(corr)$vectors[, 1:2] e1 <- x.eigen[, 1] e2 <- x.eigen[, 2] alpha <- ifelse(e1 > 0, atan(e2/e1), atan(e2/e1) + pi) corr <- corr[order(alpha), order(alpha)] }  Question: What is the interpretation of such ordering and what theory lies behind it? ## 1 Answer It is described in Michael Friendly's American Statistician paper on corrgrams, Preprint PDF here. See section on correlation ordering. Also if you look at the source of the corrgram library you will see some other potential ways to order the data as well. To describe what the code is doing in a nut-shell, the variables in the correlation matrix are ordered according to the correlations with the first and the second principle components extracted from that same correlation matrix. If you look at the Eigenvector plot in the Friendly paper (Figure 3), the code atan(e2/e1) is the angle between the ray associated with a particular variable and the horizontal axis. The variables are sorted by this angle, in a counter-clockwise order. If the whole picture were squeezed horizontally by the square root of the first eigenvalue, and vertically by the square root of the second eigenvalue (this would not change the order!), then the$x$and$y$coordinates of each ray's endpoint would be exactly the correlations of this variable with PC1 and with PC2. Again the reason for the ordering is given in the Friendly paper, but we almost always want more similar things next to more similar things (in either graphics or tables). Frequently the ordering is more informative than the numbers or the graph! Here in this example "more similar" is defined in terms of correlations to the first and the second principle components. Also note I assume the first if statement in the code prevents this ordering from occurring if the correlation matrix is not full rank. • I think your second paragraph does not describe it correctly. E.g. "the variables in the correlation matrix are ordered according to the correlation with the first principle component" -- no, not really! For each variable, correlations$c_1$and$c_2\$ with the two first PCs are computed, and then variables are plotted on a 2D plot according to these two coordinates. This is what is used to draw arrows on biplots. Then this procedure orders variables by the angle of the corresponding "biplot arrow". [Also, your second link does not work anymore.] – amoeba Feb 4 '15 at 14:28
• Maybe I'm confused amoeba, but in the biplot atan(e2/a1) is the angle between one ray and the horizontal axis (this of course can be used to figure out the angle to the vertical axis). Isn't the cosine of this angle the correlation between those variables and the first PC? (And subsequently the cosines of the angles between each ray the correlation between those measures?) I will work on the text when I get a chance (replace "ordered based on correlation" to "ordered based on the arccosines of the correlations" or something similar.) – Andy W Feb 4 '15 at 15:22
• It is unfortunate the second link is not available anymore. Wilkinson and Friendly have a paper, History of the Cluster Heat Map, which is similar in spirit but I remember the Wilkinson presentation having much more discussion and some analysis of different layouts for heatmaps. – Andy W Feb 4 '15 at 15:25
• Yes, it's the angle between a ray and the x-axis. The cosine of this angle is not exactly the correlation with the first PC (and the cosines of angles between rays are not exactly the correlations between variables), but are supposed to approximate them. Usually rays represent loadings (i.e. eigenvectors scaled by the square roots of the eigenvalues), and then e1 and e2 are exact correlations with the first/second PCs. However, my point was that the ordering is not done based on the correlation with PC1 alone! It's based on the correlations with PC1 and with PC2. – amoeba Feb 4 '15 at 15:32
• By the way, your answer could improve a lot if you included a figure (perhaps Figure 3 from Friendly's article); then the explanation becomes much more clear. Note that Friendly does not use loadings for plotting rays, but the eigenvectors themselves. However, it does not seem to matter for ordering, as using loadings would only stretch the whole arrangement horizontally/vertically. – amoeba Feb 4 '15 at 15:33