Variances explained by each feature on PC in PCA

Question

I came across this article with an associated Python codebase.

In brief, there is a section "Understanding How Features Contribute to PCs ", where...

One method for understanding which features are ‘important’ is to examine how each feature contributes to each principal component. To do this, we can take the dot product of our original data and our principal components.Assuming our data is rescaled, the relative magnitudes of its dot product with the principal components will indicate the co-linearity or correlation of individual features and PCs. In other words, if a feature is nearly co-linear with a PC, the magnitude of the dot product will be relatively large.

Found in line 156 of the Python code:

def pca_feature_correlation(X, X_pca, explained_var, features:list=None, fig_dpi:int=150, save_plot:bool=False)
...

The PC-Feature matrix is surely not a correlation matrix? perhaps a covariance matrix? Or am I missing something..?

Anyway then they go on and...

I decided to re-normalize the heatmap with the explained variance of each PC. I essentially took the dot product of our new matrix (the one above, but scaled/z-normalised) and the explained variance as a vector. This would immediately reveal not only how each feature correlates with each PC, but how they contribute to the variance in the dataset.

Found in line 192 of the Python code:

def normalize_dataframe(df, explained_var=None, fig_dpi:int=150, plot_opt:bool=True, save_plot:bool=False):

I think there is a typo there, as you can not take a dot product of your PC-Feature matrix and the variances, but just a normal multiplication. Also they apparently take the absolute value of the Z-normalised PC-Feature matrix to multiply by the variance.

But anyway...Implementing this in R:

    library(tidyverse)
    
    #Get Data
    data <- iris %>%
      select(where(is.numeric))
        
    
    #PCA
    pcaFit <- data %>%
      prcomp(scale = TRUE)
    
    #Transpose of original dataset
    tData <- data %>%
      scale() %>%
      as.matrix() %>%
      t()
    
    #Get PC data
    PCA <- pcaFit$x %>%
      as.matrix()
    
    #Dot product of transposed dataset with PCA data
    corrMatrix <- tData %*% PCA %>%
      scale() %>% #And then scaling 
      abs() #Remove negative numbers

    #              PC1       PC2        PC3       PC4
    # Sepal.Length 0.02322380 0.6025376 1.41380242 0.5224832
    # Sepal.Width  1.20005339 1.0940221 0.09737944 0.1939091
    # Petal.Length 1.24847805 0.8695742 0.40801633 1.4611018
    # Petal.Width  0.02520086 0.8269854 0.90840665 0.7447095
    
    #Multiply by variance
    frankensteinMatrix <- corrMatrix * pcaFit$sdev^2

Questions: What am I looking at with the PC-Feature corrMatrix?

What exactly do you get when you multiply the corrMatrix by the variance pcaFit$sdev^2 ? What are those numbers in that matrix?

Am aware of the following discussions:

• PCA for Feature Selection and Calculating the Total Contributions of Each Features
• Using principal component analysis (PCA) for feature selection
• PCA and variable contributions to first n dimensions
• https://stackoverflow.com/questions/12760108/principal-components-analysis-how-to-get-the-contribution-of-each-paramete

Answer 1

It is helpful to use stepwise regression (which you should avoid at all costs when predicting Y) to provide a concise explanation of what PCs are doing. This is done automatically with the R Hmisc package princmp function which also works for sparse PCs. Sparse PCs can be more interpretable because they combine variable clustering with PCA. Examples of use of princmp are here which also includes a graphic depicting PC loadings to help in the interpretation.