How does the mean influence PCA?
What happens if I use PCA on data with a mean $\ne0$?
$\begingroup$
$\endgroup$
2
-
2$\begingroup$ This question: stats.stackexchange.com/q/22329/3277 is very similar $\endgroup$– ttnphnsCommented Apr 17, 2014 at 7:57
-
$\begingroup$ Hi even if the question is very similar here the focus is on pca. The answer on this question for example give more information on pca thant the answers on the other question. I do not think that PCA can be related 100% with intercept estimation problem $\endgroup$– DonbeoCommented Apr 18, 2014 at 10:07
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
PCA is sensitive to the scaling of the variables. ... One way of making the PCA less arbitrary is to use variables scaled so as to have unit variance, by standardizing the data and hence use the autocorrelation matrix instead of the autocovariance matrix as a basis for PCA. However, this compresses the fluctuations in all dimensions of the signal space to unit variance.
Mean subtraction (a.k.a. "mean centering") is necessary for performing PCA to ensure that the first principal component describes the direction of maximum variance. If mean subtraction is not performed, the first principal component might instead correspond more or less to the mean of the data. A mean of zero is needed for finding a basis that minimizes the mean square error of the approximation of the data.
From PCA, Further considerations.
This way PCA on covariance matrix of non-standardized data will be affected by the non-zero means present. Standardizing the data or explicit use of correlation matrix will cancel this effect (though loosing relative scales of homogenous variables may be considered harmful).
-
2$\begingroup$ PCA of the correlation matrix, $\mathbf{R}$ is insensitive to linear transformations of the data matrix $\mathbf{X}$. Both the eigenvalues and eigenvectors of $\mathbf{X}$ are equal to the eigenvalues and eigenvectors of $a\mathbf{X} + b$ for all values of $0 < a$ and $-\infty < b <\infty$. $\endgroup$– AlexisCommented Apr 25, 2014 at 22:34
-
$\begingroup$ @Alexis, interesting note. Does the updated post look more comprehensive? $\endgroup$ Commented Apr 26, 2014 at 8:59