# What is percentage of variance in PCA? [duplicate]

I am trying to understand Principal Component Analysis (PCA). I found a webpage on PCA that introduces it and the concept of the percentage of variance. However, I am very confused about what "Percentage of Variance" (POV) means. Here are the questions I have:

1. Is there a formal definition or mathematics formula to define "percentage of variance"?
2. Do we only use POV for calculating PCA? Is POV used somewhere else?
3. Is POV the same as "Proportion of Variance Explained"? (There are many similar terms online which really confuse me.)
• Cassie, PCA works with variance-covariance matrix, where variances are its diagonal. Sum of variances is overall multivariate variability. It is this POV is computed of. For more, see here Commented Jul 9, 2012 at 6:52
• The link you provided is great. Thus, intuitively, when a component has accounted for larger percentage of variance, can I say the component is more important then others ? Moreover, assume I develop a new variable selection method to select only 5 out of 10 original variables. Without using PCA to find eigenvectors, assume I just calculate cov matrix for these 5 variables. Then, can I calculate POV of these 5 variables in the same way ? Or POV can only be applied on eigenvalues ? Thanks, Commented Jul 10, 2012 at 1:40
• Using PCA for feature selection is a little bit different, because the eigenvectors usually do not correspond to input features but rather to a linear combination of these features. Therefore, you can use PCA to reduce the dimensionality of your dataset while still preserving most of the variance by creating a new feature space based on the top $k$ eigenvectors (according to their eigenvalues). However, the "new" features will be different from the original ones.
– sebp
Commented Jul 10, 2012 at 8:17

I assume they are referring to the eigenvalues of the corresponding eigenvectors. The eigenvalues in PCA tell you how much variance can be explained by its associated eigenvector. Therefore, the highest eigenvalue indicates the highest variance in the data was observed in the direction of its eigenvector. Accordingly, if you take all eigenvectors together, you can explain all the variance in the data sample. Instead of using the absolute value of variance explained, as indicated by the eigenvalue, you can also get relative numbers by first summing up all eigenvalues and then divide an eigenvalue $\lambda_i$ by this sum $$\frac{\lambda_i}{\sum_{i=1}^n \lambda_i} .$$ This way you end up with a "percentage of variance" for each eigenvector.
• Just a clarification: if $X_1,\ldots,X_p$ are the original random variables, then $\sum_{i=1}^p\lambda_i=\sum_{i=1}^p{\rm Var}(X_i)=tr(\Sigma)$ where $\Sigma$ is the covariance matrix. Commented Jul 9, 2012 at 8:51