Is it possible to visualize the output of Principal Component Analysis in ways that give more insight than just summary tables? Is it possible to do it when the number of observations is large, say ~1e4? And is it possible to do it in R [other environments welcome]?
The biplot is a useful tool for visualizing the results of PCA. It allows you to visualize the principal component scores and directions simultaneously. With 10,000 observations you’ll probably run into a problem with over-plotting. Alpha blending could help there.
Here is a PC biplot of the wine data from the UCI ML repository:
The points correspond to the PC1 and PC2 scores of each observation. The arrows represent the correlation of the variables with PC1 and PC2. The white circle indicates the theoretical maximum extent of the arrows. The ellipses are 68% data ellipses for each of the 3 wine varieties in the data.
I have made the code for generating this plot available here.
A Wachter plot can help you visualize the eigenvalues of your PCA. It is essentially a Q-Q plot of the eigenvalues against the Marchenko-Pastur distribution. I have an example here: There is one dominant eigenvalue which falls outside the Marchenko-Pastur distribution. The usefulness of this kind of plot depends on your application.
You could also use the psych package.
This contains a plot.factor method, which will plot the different components against one another in the style of a scatterplot matrix.