0
$\begingroup$

I came across this article with an associated Python codebase.

In brief, there is a section "Understanding How Features Contribute to PCs ", where...

One method for understanding which features are ‘important’ is to examine how each feature contributes to each principal component. To do this, we can take the dot product of our original data and our principal components.Assuming our data is rescaled, the relative magnitudes of its dot product with the principal components will indicate the co-linearity or correlation of individual features and PCs. In other words, if a feature is nearly co-linear with a PC, the magnitude of the dot product will be relatively large.

enter image description here

Found in line 156 of the Python code:

def pca_feature_correlation(X, X_pca, explained_var, features:list=None, fig_dpi:int=150, save_plot:bool=False)
...

The PC-Feature matrix is surely not a correlation matrix? perhaps a covariance matrix? Or am I missing something..?

Anyway then they go on and...

I decided to re-normalize the heatmap with the explained variance of each PC. I essentially took the dot product of our new matrix (the one above, but scaled/z-normalised) and the explained variance as a vector. This would immediately reveal not only how each feature correlates with each PC, but how they contribute to the variance in the dataset. enter image description here

Found in line 192 of the Python code:

def normalize_dataframe(df, explained_var=None, fig_dpi:int=150, plot_opt:bool=True, save_plot:bool=False):

I think there is a typo there, as you can not take a dot product of your PC-Feature matrix and the variances, but just a normal multiplication. Also they apparently take the absolute value of the Z-normalised PC-Feature matrix to multiply by the variance.

But anyway...Implementing this in R:

    library(tidyverse)
    
    #Get Data
    data <- iris %>%
      select(where(is.numeric))
        
    
    #PCA
    pcaFit <- data %>%
      prcomp(scale = TRUE)
    
    #Transpose of original dataset
    tData <- data %>%
      scale() %>%
      as.matrix() %>%
      t()
    
    #Get PC data
    PCA <- pcaFit$x %>%
      as.matrix()
    
    #Dot product of transposed dataset with PCA data
    corrMatrix <- tData %*% PCA %>%
      scale() %>% #And then scaling 
      abs() #Remove negative numbers

    #              PC1       PC2        PC3       PC4
    # Sepal.Length 0.02322380 0.6025376 1.41380242 0.5224832
    # Sepal.Width  1.20005339 1.0940221 0.09737944 0.1939091
    # Petal.Length 1.24847805 0.8695742 0.40801633 1.4611018
    # Petal.Width  0.02520086 0.8269854 0.90840665 0.7447095
    
    #Multiply by variance
    frankensteinMatrix <- corrMatrix * pcaFit$sdev^2

Questions: What am I looking at with the PC-Feature corrMatrix?

What exactly do you get when you multiply the corrMatrix by the variance pcaFit$sdev^2 ? What are those numbers in that matrix?

Am aware of the following discussions:

$\endgroup$

1 Answer 1

1
$\begingroup$

It is helpful to use stepwise regression (which you should avoid at all costs when predicting Y) to provide a concise explanation of what PCs are doing. This is done automatically with the R Hmisc package princmp function which also works for sparse PCs. Sparse PCs can be more interpretable because they combine variable clustering with PCA. Examples of use of princmp are here which also includes a graphic depicting PC loadings to help in the interpretation.

$\endgroup$
2
  • $\begingroup$ I have a naive question on this, as I am more a biologist than a statistician 1) While A normal PCA does not allow non-zero loadings (as opposed to a sparse PC), we can still get the variance percentages say from a prcomp object and calculate the %variance contributed by each feature. What is the advantage of a princmp vs doing this? Is it for the step-wise regression? 2) Is the underlying calculation of what features contribute to a principle component determined by a stepwise regression AFTER PC determination or are the PCs themselves derived from the correlation of the features? $\endgroup$
    – Rover Eye
    Commented Sep 23, 2023 at 13:50
  • 1
    $\begingroup$ It's just the stepwise part to try to get a concise PC description. $\endgroup$ Commented Sep 23, 2023 at 22:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.