1
$\begingroup$

I want to produce a scree plot to assess if there is an 'elbow' in the eigenvalues to aid in my identification of the number of PCs to retain. However, upon reading further into the topic, I realised that the eigenvalues are only plotted when the correlation matrix is used and that the log of the eigenvalues is required if the PCA used the covariance matrix.

I'm not entirely clear on the difference between these two, but I used 'pca' in matlab to carry out my analysis and it says on the documentation that the 'latent' output (i.e. the eigenvalues) are 'the eigenvalues of the covariance matrix of X' (X is the data).

I normalised my data using zscore prior to executing pca. Does that make a difference? My ultimate question is: can I use the eigenvalues in my scree plot, or do I have to get the log of them to plot?

$\endgroup$
2
  • 1
    $\begingroup$ In regards to the question in the title: The function pca in MATLAB uses the SVD of the centred dataset to perform PCA; this excellent thread elucidates the relation between the two. Using the SVD correspond to using the covariance matrix, not the correlation matrix. $\endgroup$
    – usεr11852
    Oct 26, 2016 at 21:50
  • 1
    $\begingroup$ And to mention out the obvious: if one z-scores the data and then uses the covariance matrix for PCA, the results will be equivalent to using the correlation matrix of the original data... After all, cov(zscore(A)) - corr(A) should be zero to numerical precision... (where A is the dataset matrix) $\endgroup$
    – usεr11852
    Oct 26, 2016 at 22:18

2 Answers 2

2
$\begingroup$

In regards to the question in the title: The function pca in MATLAB uses the SVD of the centred dataset to perform PCA; this excellent thread elucidates the relation between the two. Using the SVD corresponds to using the covariance matrix, not the correlation matrix.

Having said that and to answer the main question of post: if one z-scores the data and then uses the covariance matrix for PCA, the results will be equivalent to using the correlation matrix of the original data. This can be easily seen by computing the difference: cov(zscore(A)) - corr(A) which should be zero to numerical precision (where Ais the dataset matrix).

So yes, there will be a difference if you use the correlation-based instead of the covariance-based PCA methodology; if you $z$-score your dataset though the two methodologies will give equal results. In general, I would recommend you $z$-scale your variables before doing PCA, especially if they are measured in different scales. Otherwise the differences in their magnitudes can potentially dominate the subsequent eigenanalysis (and the interpretation of the final results). This topic is explored in more detail in this thread on doing PCA on correlation or covariance?

$\endgroup$
1
  • $\begingroup$ +1 and I took the liberty to add one more link. $\endgroup$
    – amoeba
    Oct 29, 2016 at 12:32
1
$\begingroup$

Edit: I defer questions about correlation matrix vs covariance matrix to this thread where it was discussed at great length.

To your ultimate question, you can certainly plot the explained variance vs. # of dimensions (scree plot) without having to log anything:

enter image description here

Feel free to play around with the code (in Python):

from sklearn import datasets
from sklearn.preprocessing import StandardScaler
import numpy as np
from numpy import linalg as la
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline

iris = datasets.load_iris()
# mean center the data
mean_centered = StandardScaler().fit_transform(iris.data)

# this uses covariance matrix but you can use correlation
cov_matrix = np.cov(mean_centered.T)

# get your eigenvalues. No need to log them.
eig_vals = la.eig(cov_matrix)[0]

total_variance = np.sum(eig_vals)
# the cumulative sum of the eigenvalues is your % variance explained
cumulative_variance_explained = np.cumsum(eig_vals)/total_variance

plt.bar(np.arange(1,5), cumulative_variance_explained,align ='center',alpha=0.6,width=0.5)
plt.ylim(0,1.2)
plt.xlim(0,5)
plt.title("Scree Plot for Iris Dataset")
plt.xlabel("Number of Principle Components")
plt.ylabel("% Variance Explained")
pass
$\endgroup$
4
  • $\begingroup$ Which part? Eigenvalues being slightly different or the interpretation of the results or both? $\endgroup$
    – ilanman
    Oct 26, 2016 at 20:42
  • $\begingroup$ ok. I was speaking more from experience. I'll edit that part out. $\endgroup$
    – ilanman
    Oct 26, 2016 at 20:59
  • 1
    $\begingroup$ Maybe you wanted to say that the results from correlation- and covariance-based methodology are numerically equally if you z-scale your data beforehand? $\endgroup$
    – usεr11852
    Oct 26, 2016 at 22:21
  • $\begingroup$ Yes though you stated it in a more succinct and precise way $\endgroup$
    – ilanman
    Oct 27, 2016 at 0:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.