Is it possible to visualize the output of Principal Component Analysis in ways that give more insight than just summary tables? Is it possible to do it when the number of observations is large, say ~1e4? And is it possible to do it in R [other environments welcome]?
-
2$\begingroup$ A few questions: How many components do you have? Besides the sample size, is there anything that makes the display of this PCA output need to be different from the display of other continuous variables that one might be dealing with? Are you trying to contrast scores of different groups, and if so how many? Generally, what are you hoping to achieve with your displays? $\endgroup$– rolando2Mar 3, 2011 at 23:26
3 Answers
The biplot is a useful tool for visualizing the results of PCA. It allows you to visualize the principal component scores and directions simultaneously. With 10,000 observations you’ll probably run into a problem with over-plotting. Alpha blending could help there.
Here is a PC biplot of the wine data from the UCI ML repository:
The points correspond to the PC1 and PC2 scores of each observation. The arrows represent the correlation of the variables with PC1 and PC2. The white circle indicates the theoretical maximum extent of the arrows. The ellipses are 68% data ellipses for each of the 3 wine varieties in the data.
I have made the code for generating this plot available here.
-
5
-
$\begingroup$ This is by far the prettiest biplot I have ever seen, +1 long time ago. I have a question about scaling the arrows (loadings) that you chose: what is the radius of the white circle? It is not equal to $1$ (maximal value for a correlation), so some scaling must have been done. Is it arbitrary (to make the circle and the arrows large enough to be nicely seen), or is there some logic behind the scaling choice? $\endgroup$– amoebaJan 14, 2015 at 22:17
-
$\begingroup$ @amoeba The radius of the circle corresponds to the maximum possible length of the arrows. Say V is a $p \times 2$ matrix with each column corresponding to principal component eigenvectors (chosen to be orthonormal). Then each arrow in the biplot corresponds to a row of $V$. The Euclidean norm of each row of V ranges between 0 and 1, because those are the square roots of the diagonal entries of $VV^T$ which is a projection matrix. The circle provides a relative scale for the arrows, because the arrows and the PC scores (the points in the bipolar) are not on the same scale. $\endgroup$– vqvJun 27, 2015 at 0:43
-
$\begingroup$ Thank you, but this much I understand. My question is about how you choose the relative scale of the arrows and the PC scores. The circle has a radius of approximately 3.7, which is then obviously the scaling factor (as maximum possible length of a row in $V$ is 1). So why 3.7? $\endgroup$– amoebaJun 27, 2015 at 13:12
A Wachter plot can help you visualize the eigenvalues of your PCA. It is essentially a Q-Q plot of the eigenvalues against the Marchenko-Pastur distribution. I have an example here: There is one dominant eigenvalue which falls outside the Marchenko-Pastur distribution. The usefulness of this kind of plot depends on your application.
-
8$\begingroup$ It would be helpful to know more here (perhaps some additional explication &/or some useful links). What is the Marchenko-Pastur distribution? How does it relate to PCA? What does it mean for your results if it holds or does not? (etc) $\endgroup$ Feb 2, 2014 at 5:13
You could also use the psych package.
This contains a plot.factor method, which will plot the different components against one another in the style of a scatterplot matrix.