In the textbook “Principal Component Analysis” Jolliffe (§9.2) suggests the following method for variable reduction:

When the variables fall into well-defined clusters, there will be one high-variance PC and, except in the case of 'single-variable' clusters, one or more low-variance PCs associated with each cluster of variables. Thus, PCA will identify the presence of clusters among the variables, and can be thought of as a competitor to standard cluster analysis of variables. [...] Identifying clusters of variables may be of general interest in investigating the structure of a data set but, more specifically, if we wish to reduce the number of variables without sacrificing too much information, then we could retain one variable from each cluster.

I am trying to apply this idea of variable reduction to a problem which involves differentiating two substances based on their spectrum (composed of 1350 variables or wavelengths):

enter image description here

And here is the result of the PCA that separates the two substances by using all 1350 variables (plot of the measurements with respect to the first two PCs):

enter image description here

Unfortunately, there is no further elucidation in the textbook. So, is Jolliffe suggesting that we should plot the loadings at each variable for two PCs and look for clusters there?

If so, this is what I obtained:

enter image description here

Have I followed Jolliffe's method correctly? Because I don't really see any well-defined clusters here to choose variables from.

Any explanation is greatly appreciated.

  • $\begingroup$ I think that you should identify the clusters on the first plot, and it is clear that the red substance lies above the x axis in this projected space while the blue substance lies below the x axis. $\endgroup$ – boomkin Jan 4 at 14:06
  • $\begingroup$ But that plot doesn't tell us anything about variables — each dot corresponds to a case (i.e., samples that were measured). Jolliffe says we need a plot where we can potentially see clusters of variables... $\endgroup$ – Merin Jan 4 at 14:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.