1
$\begingroup$

I am getting into and trying to learn how to use principal component analyses (PCA), and got stuck on a few things that I thought someone here might be able to help me with.

What I am trying to do: I have a data set of a number of animal individuals. For each individual I have six different measurements (eg. weight, height, length etc.). What I want to do is to combine these six measurements into one size variable to represent all measurements, and I am doing so by carrying out a PCA.

As I've understood it, the first principal component (PC1) is always the best one / the one that explains most of the variance.

So my questions are then:

  • When I carry out a PCA on my data set, PC1 only stands for about 40% and PC2 for 30% of the variance. Does that mean I can't use only PC1 as a variable for my size measure? Is it possible to combine both PC1 and PC2 into one variable? If yes, how? Simply by adding them up or does that not work statistically? Also:

  • How much of the variance does PC1 have to explain for it to be used as a good representation, as a size variable in my case? Is 60% enough or does one want it to be over 80%?

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Welcome to Cross Validated! Why do you want to combine the variables instead of using two variables? (In fact, why run the PCA at all?) Since standard math allows you to add two vectors, so you must have a goal in mind for adding these vectors that you’re unsure if such addition accomplishes. What is that goal? $\endgroup$
    – Dave
    Commented Nov 3, 2023 at 10:43
  • 1
    $\begingroup$ By its very construction, PC1 is a linear combination with the greatest possible proportion of variance: you can't increase that by combining PC1 with any other linear combination of the data, such as other PC's. With morphometric measurements the first thing to consider -- really, even before looking at the data -- is applying nonlinear transformations to make the units commensurable: a cube root for weights and volumes, a square root for surface areas. That almost always is an improvement. $\endgroup$
    – whuber ♦
    Commented Nov 3, 2023 at 12:07
  • $\begingroup$ Depending on your implementation, the PCs are probably orthogonal to one another (completely independent and uncorrelated). To add together such values would tend toward creating random noise rather than any better indicator of something. $\endgroup$
    – rolando2
    Commented Apr 24 at 10:29

1 Answer 1

Reset to default
1
$\begingroup$

First, it sounds like you want factor analysis not PCA. You have a latent variable (overall size). Finding latent factors is the goal of factor analysis.

My favorite professor in grad school used to say:

If you're not surprised, you haven't learned anything.

The fact that PC 1 only accounts for 40% of the variance means that something else is going on. That is a surprise. Your goal is to figure out what that is. Throwing that information away by combining the two PCs seems like a really bad idea.

I don't know what you mean by "not work statistically".

What is a "good" representation is really up to you to decide. It's field dependent and also depends on the number of variables and what they are. With only six variables and given the ones you've listed (and assuming the others are similar) I am very surprised that one factor is only 40% of the variance.

Very surprised means that you either a) Did something wrong or b) Found something really interesting. You don't want to ignore either possibility.

Improve this answer
$\endgroup$
4
  • 2
    $\begingroup$ Check out sparse principal component analysis, which is akin to doing variable clustering and summarizing each cluster with $PC_1$. A case study is here. $\endgroup$
    – Frank Harrell
    Commented Nov 3, 2023 at 11:44
  • $\begingroup$ @FrankHarrell Thanks. I never heard of that. $\endgroup$
    – Peter Flom
    Commented Nov 3, 2023 at 11:45
  • $\begingroup$ Sparse PCs are typically more interpretable than regular PCs, and $PC_1$ explains more of the system variance after confining the set on which $PC$ is computed to a set of already correlated variables. $\endgroup$
    – Frank Harrell
    Commented Nov 3, 2023 at 12:07
  • $\begingroup$ With objective measurements (negligible subjectivity or measurement error or any kind) PCA is a fine alternative to factor analysis. $\endgroup$
    – rolando2
    Commented Apr 24 at 10:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.