5
$\begingroup$

What is a good, structured and reproducible method to name selected components or factors in principal component analysis (PCA)? Clearly, the sign and magnitude of the entries in the eigenvectors are key, but how do you go from there to a qualitative, solid descriptor or name?


BACKGROUND (skip unless interested in soccer): It turns out that you (meaning, 'I') barely make it through getting a shaky understanding of dimensionality reduction in data with correlated variables; fight your way through the linear algebra to understand (on a good day) the change in coordinates, the rotation of the data cloud and other arcane concepts, in great measure thanks to herculean didactic efforts directed to grandmothers, and other family members (here). Finally, the R code also falls in place (princomp v prcomp), and you are ready to select the main factors or components among the eigenvectors of the covariance matrix. You get the output of the summary, generate a biplot and you are done, hoping that somebody smarter can get through the shock of overlapping info on the biplot, and squeeze some meaningful information that can be, yes, explained to your mother, grandmother or to your son. But you can't just print the biplot - you are at the dinner table...

So what to do? How to give meaningful name to PC1 and PC2? Do they need a name? They should have a name, because just going though the components in the corresponding eigenvectors doesn't seem like a great, impacting way of getting a paper published. So what are the guidelines or principles to assign these names?

Here's an example for soccer fans with kids with questions. You get the Economist, and finally understand what makes tiki-taka the best soccer ever played. That is if you know that ABCB means (from watching the games) that the ball goes from Busquets to Xavi, from Xavi to Iniesta, and from Iniesta back to Xavi, for example. And that in ABAC the A may be Messi looking for a wall before giving an assistance for goal. ABAB stands for two soccer geniuses hugging the ball while the defenders give up in frustration. So now (sorry baseball and football fans) "Principal Component 1" in the opening plot makes a lot of sense. But what happens if you are not a Barça fan... you are then at the mercy of the explanation of "Principal Component 1": "a much greater propensity to play ABAC and ABCB passing sequences, as well as ABAB". And because it's the Economist, there is some further explaining. But still... this was the first time I ever heard of PCA, and couldn't get faster to Wikipedia. Shouldn't there be an effort made to name this component or factor something that sounds like a new meta-variable - something like, "back-and-forth passing" (not great, I know)?

The same thing tends to happen in other instances in which after the mathematical effort is carried out, the assignment of qualitative descriptors or names to the PC's is not compelling. Without nice definitions (or ideally, names) for the principal components, it is difficult to grasp what they mean, and how they are a weighted linear composition of the initial variables.

Here's a plot in an otherwise phenomenal post on-line (I will hold off on crediting it because of what follows) at the end of a PCA on different models of cars in the year 2004 - basically data on technical specs, price, etc:

enter image description here

Very hard, isn't it, to go for any cars other than luxury German models?

$\endgroup$
  • 4
    $\begingroup$ PCs are just some derived variables, the axes/directions in the multidimensional space of the data. If I give you an arbitrary scatterplot of some data poins where all the points are labeled but the axes are not labeled - will you have difficulty inventing the missing labels? $\endgroup$ – ttnphns Jul 18 '15 at 5:55
8
$\begingroup$

You focus on "naming", but I would say that the real problem is understanding what principal components mean. You are right: this is an art. It often turns out that they are very difficult to interpret, hence all the attempts (especially in factor analysis literature and practice) to rotate the components/factors in order to achieve "simple structure", i.e. structure that will be easier to interpret (see my answer here).

I don't know where you took your figure from, but this dataset was nicely analyzed in Cosma Shalizi's lecture notes on PCA, and I quote from page 7:

This [matrix of eigenvectors] says that all the variables except the gas-mileages have a negative projection on to the first component. This means that there is a negative correlation between mileage and everything else. The first principal component tells us about whether we are getting a big, expensive gas-guzzling car with a powerful engine, or whether we are getting a small, cheap, fuel-efficient car with a wimpy engine.

The second component is a little more interesting. Engine size and gas mileage hardly project on to it at all. Instead we have a contrast between the physical size of the car (positive projection) and the price and horsepower. Basically, this axis separates mini-vans, trucks and SUVs (big, not so expensive, not so much horse-power) from sports-cars (small, expensive, lots of horse-power).

Once you understood that, you can look for good names.

$\endgroup$
  • $\begingroup$ Ah, okay. Apologies if you did not want to reveal Cosma's analysis to other readers. What's a "qplot" by the way? What you inserted in your post is called a "biplot". $\endgroup$ – amoeba Jul 18 '15 at 13:54
  • $\begingroup$ Yes. I did read the description on the plot, and the article helped me a lot, which is why I was hesitant to quote it, given that the plot (replicated using R after downloading the data) was attempted to make the point that sometimes PCA, from the lack of good summarizing names to the busy biplots, leaves amateurs / beginners like myself between fascinated and speechless. Thank you for taking the post seriously. It stemmed from my difficulties in trying to reconcile the math with the ultimate results, and sifting though papers looking for nice legends to the plots $\endgroup$ – Antoni Parellada Jul 18 '15 at 13:56
  • $\begingroup$ biplot - I don't know where 'qplot' came from. Here is the code I used. Thank, again! $\endgroup$ – Antoni Parellada Jul 18 '15 at 13:57
  • $\begingroup$ Good answer, but I would go further: It is an art that is very often bound to fail. There is no guarantee whatsoever that principal components can be meaningfully interpreted (except maybe for the first one). The reason is that interpretations depend on model assumptions, but PCA is a procedure that is not based on a model – unless you happen to have a model where the data is a linear combination of uncorrelated signals with orthogonal distribution across original measurements, which is rare. $\endgroup$ – A. Donda Jul 18 '15 at 23:05
  • $\begingroup$ @A.Donda: "interpretations depend on model assumptions" - I don't follow. Why would I need to have a model or to make any assumptions when interpreting a principal component? - aren't I just describing a linear combination of the original variables? $\endgroup$ – Scortchi Jul 31 '15 at 9:38

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.