In PCA, does significance/importance of a PC correlate with explained variance?

Question

For PCA eigenvalues

S = [1.74 1.45 0.93 0.77 0.50 0.30 0.25 0.13 0.00 ...]

where the 2 first PCs explain more than 50% of the variance, is PC1 the most important/significant factor?

Or as the first 2PCs explain 50% of the variance the rest 6 PCs combined are equally significant as the first two? If not, is the PC1 is the most significant part of the variance for the data under investigation?

Essentially, would it make sense to refer to PC1 representation of the data as the most important for the system under investigation?

Answer 1

1. That a variable has high variance does not imply that it's objectively important!
2. That a variable has low variance does not imply that it's objectively unimportant!

Trivial example: hair color and blood loss in an emergency room

Imagine we have data collected from an emergency room:

• $x_1$ measures hair color.
• $x_2$ measures blood loss.

Just for hypothetical purposes, let's say the covariance matrix is: $$\Sigma = \left[ \begin{array}{cc} 100 & 0 \\ 0 & 1\end{array} \right] $$

• There's tons of variation in hair color $x_1$.
• On the other hand, blood loss is quite rare. Expected squared deviations from mean blood loss (i.e. the variance) is quite low.

You can instantly see where this is going. An eigenvalue decomposition is:

$$ \left[ \begin{array} {cc} 1 & 0 \\ 0 & 1 \end{array} \right] \left[ \begin{array} {cc} 100 & 0 \\ 0 & 1 \end{array} \right] \left[ \begin{array} {cc} 1 & 0 \\ 0 & 1 \end{array} \right] $$

The first principal component will be $\left[ \begin{array}{c} 1 \\ 0 \end{array} \right]$, the eigenvector associated with the largest eigenvalue.. Hair color describes most of the variation in our data, but that's almost certainly irrelevant in this context!

Disclaimer/note: I have no medical training, and undoubtedly the medical specifics of this hypothetical are all screwed up. Don't take that part literally. The point is about Principal Component Analysis (PCA).

Answer 2

The PCA decomposition explains the 'total variance' for your data set. Therfore, it really depends on the question you are asking. If the majority of variance in your data is caused by the effect under investigation then discarding of lower variance PCs can be seen as denoising.

But if your data set has variance caused by something irrelevant for the question you are asking this variance is also covered in PCA and might contribute stronger to the total variance then the effect you are interested in.

Interpreting the loadings of the PCA often helps to find out how many PCs are important for your data set.

Your S looks like the list of eigenvalues corresponding to the PCs, Scores in contrast can be seen as a meassure of how much each PC contributes to each individual object in your data set...

So to answer the question as it is asked - scores do not correlated with the importance of the PC - the eigenvalues can correlate depending on the structure of your data if there are no confounders.