In brief, there is a section "Understanding How Features Contribute to PCs ", where...
One method for understanding which features are ‘important’ is to examine how each feature contributes to each principal component. To do this, we can take the dot product of our original data and our principal components.Assuming our data is rescaled, the relative magnitudes of its dot product with the principal components will indicate the co-linearity or correlation of individual features and PCs. In other words, if a feature is nearly co-linear with a PC, the magnitude of the dot product will be relatively large.
Found in line 156 of the Python code:
def pca_feature_correlation(X, X_pca, explained_var, features:list=None, fig_dpi:int=150, save_plot:bool=False) ...
The PC-Feature matrix is surely not a correlation matrix? perhaps a covariance matrix? Or am I missing something..?
Anyway then they go on and...
I decided to re-normalize the heatmap with the explained variance of each PC. I essentially took the dot product of our new matrix (the one above, but scaled/z-normalised) and the explained variance as a vector. This would immediately reveal not only how each feature correlates with each PC, but how they contribute to the variance in the dataset.
Found in line 192 of the Python code:
def normalize_dataframe(df, explained_var=None, fig_dpi:int=150, plot_opt:bool=True, save_plot:bool=False):
I think there is a typo there, as you can not take a dot product of your PC-Feature matrix and the variances, but just a normal multiplication. Also they apparently take the absolute value of the Z-normalised PC-Feature matrix to multiply by the variance.
But anyway...Implementing this in R:
library(tidyverse) #Get Data data <- iris %>% select(where(is.numeric)) #PCA pcaFit <- data %>% prcomp(scale = TRUE) #Transpose of original dataset tData <- data %>% scale() %>% as.matrix() %>% t() #Get PC data PCA <- pcaFit$x %>% as.matrix() #Dot product of transposed dataset with PCA data corrMatrix <- tData %*% PCA %>% scale() %>% #And then scaling abs() #Remove negative numbers # PC1 PC2 PC3 PC4 # Sepal.Length 0.02322380 0.6025376 1.41380242 0.5224832 # Sepal.Width 1.20005339 1.0940221 0.09737944 0.1939091 # Petal.Length 1.24847805 0.8695742 0.40801633 1.4611018 # Petal.Width 0.02520086 0.8269854 0.90840665 0.7447095 #Multiply by variance frankensteinMatrix <- corrMatrix * pcaFit$sdev^2
What am I looking at with the PC-Feature
What exactly do you get when you multiply the
corrMatrix by the variance
pcaFit$sdev^2 ? What are those numbers in that matrix?
Am aware of the following discussions:
- PCA for Feature Selection and Calculating the Total Contributions of Each Features
- Using principal component analysis (PCA) for feature selection
- PCA and variable contributions to first n dimensions