2
$\begingroup$

What is the motivation for demeaning the data when doing PCA. I've been told to do it, but I've never heard a good and/or intuitive reason for it. Is this a case where doing it just makes the math easier, or is it more like: if you don't do it, you'll get the wrong answer.

This is something I've been wondering about for a while.

One other question: when you demean the data, do you also need to normalize it (i.e., divide by standard deviation)?

$\endgroup$
  • 1
    $\begingroup$ Might I suggest you read over some of the better posts on pca on this site? I believe your questions are addressed in many of them, at least in passing. For instance, your second question concerns the distinction between performing PCA with a covariance matrix or the correlation matrix. $\endgroup$ – whuber Feb 12 '14 at 17:28
  • $\begingroup$ The duplicate addresses the centering ("demeaning") question. $\endgroup$ – whuber Feb 12 '14 at 17:51
9
$\begingroup$

You can certainly do it, and the calculations work.

But of course you get different results.

The difference is between $\mathbf X^T \mathbf X$, the variance-covariance matrix (which is proportional to $\mathbf X^T \mathbf X$ if $\mathbf X$ is mean centered) , and the correlation matrix (for mean centering + variance scaling).

Whether using the data without centering is sensible depends on your data and application:
E.g. for my spectroscopic data, the practical result of not centering is usually that the first component will be the mean (and thus cover multiplicative changes of the mean spectrum) as the changes within the data are usually small compared to the mean.

In this situation, doing a PCA without centering allows me to capture changes e.g. by intensity fluctuations in the light source/uneven illumination or due to slight defocusing. Often, I want to ignore those for my further analysis, and in that case it is good to have them "together" in the 1st component instead of spread out over several components. Note that the difference is not only that the center is "prepeded": changes in the data that can be described by multiples of the mean will be captured by the "center component" if the data is mean centered, these changes will be captured by other components.

However, if I want to quantitate concentrations, these changes are often needed and centering to the mean or another from a physico-chemical point of view even more suitable center (e.g. blank spectrum) would be preferred.

The other extreme would be the situation where the mean is negligible compared to the changes. In that case, the first components won't change much. And, of course, there is the intermediate range where you can neiter assume the center to dominate $\mathbf X^T \mathbf X$, nor assume it negligible: in that case the results may change very much.

$\endgroup$
  • 2
    $\begingroup$ Everything here is correct and nicely explained. I am a little concerned, though, that you might be leaving the impression that essentially the only difference between centered and non-centered PCA lies in the first PC (which would be a reasonable conclusion to draw). I just want to point out this is decidedly not the case in general, as explained in my answer at stats.stackexchange.com/a/35190. $\endgroup$ – whuber Feb 12 '14 at 17:54
  • 1
    $\begingroup$ @whuber: thanks - I clarified the answer to point this out more strongly and also that this is one extreme case. $\endgroup$ – cbeleites supports Monica Feb 12 '14 at 18:09

Not the answer you're looking for? Browse other questions tagged or ask your own question.