I am a newbie to principal component analysis (PCA). I will have to do PCA to data sets consisting of count statistics: all data are positive integers.

Before PCA the data needs to be normalized. It is more or less standard to do that by subtracting the mean and dividing by the standard deviation in the variable over the sample set. I wonder whether this is appropriate for data sets that are very skewed.

  • $\begingroup$ As the A in PCA means "analysis", the expression "PCA analysis" reads awkwardly. I've edited it out. $\endgroup$ – Nick Cox Jan 14 '14 at 14:51
  • 3
    $\begingroup$ @NickCox While it does read awkwardly if you expand out PCA, some sources state it is acceptable to 'duplicate' the last word like that, as with PIN number, ATM machine, HIV virus, ISBN number, LCD display, SAT test. Others say it's a mistake, but I think that itself is an error -- but even if it isn't, that horse sailed long ago. $\endgroup$ – Glen_b Jan 14 '14 at 23:41
  • $\begingroup$ I don't think think statistical people around me ever say "PCA analysis" and I'm certainly recommending against using that in writing. The cross-reference should interest others interested in these details, so thanks! $\endgroup$ – Nick Cox Jan 15 '14 at 3:19

First off, be aware that the term "normalize" is ambiguous within statistical science. You apply it to scaling by (value $-$ mean) / standard deviation, which is commonly also described as standardization. But it is also often applied to transformations that produce versions of a variable that are more nearly normal (Gaussian) in distribution. Yet again, a further use is that of scaling to fit within a prescribed range, say $[0, 1]$.

Standardization itself does not affect how far a distribution is normal, as it is merely a linear transformation, and skewness and kurtosis (for example), and more generally all measures of distribution shape, remain as they were.

As for principal component analysis (PCA), prior standardization is common, indeed arguably essential, whenever the individual variables are measured using different units of measurement. Conversely, PCA without standardization can make sense so long as all variables are measured in the same units. The difference corresponds to basing PCA on the correlation matrix (prior standardization) and on the covariance matrix (no prior standardization). Without standardization, PCA results are inevitably dominated by the variables with highest variance; if that is desired (or at worst unproblematic), then you will not be troubled.

Other way round, all variables being standardized gives them all, broadly speaking, the same importance; and even that could be wrong, or not what you most want. For example, the variable with the least variance and that with the most will end up on the same scale and with equal weight. Only rarely does that match what a researcher most needs, although it can be hard to build in what is needed without subjectivity or circularity. In practice, PCA seems most successful when the input variables have a strong family resemblance and least successful when the researcher inputs a mishmash of quite different variables, as say different social, economic or demographic characteristics of countries or other political units. PCA is not a washing machine; the dirt is not removed, but just redistributed.

If skewness is very high, you have a choice. Often results will be clearer if PCA is applied to transformed variables. For example, the effects of outliers or extreme data points will often be muted when variables are transformed. Conversely, PCA as a transformation technique does not depend on, or assume, that any (let alone all) of the variables fed to it being normally distributed.

In abstraction, it is difficult to advise in detail, but it will often be sensible to apply PCA both to the original data when highly skewed and to transformed data, and then to report either or both results, depending on what is helpful scientifically or substantively.

PCA itself is indifferent to whether variables are transformed in the same way, or indeed to whether some variables are transformed and others are not. Whenever it makes sense, there is some appeal in transforming variables in the same way, but this is perhaps more a question of taste than of technique.

As a simple example, if several variables are all measures of size in some sense, then skewness is very likely. Transforming all variables by taking logarithms (so long as all values are positive) will then often be valuable as a precursor to PCA, but neither analysis should be thought of as "correct"; rather they give complementary views of the data.

Note 1: I rather doubt that you "have to" do PCA unless you are committed to some exercise as part of a course of study. It seems very likely that some kind of Poisson modelling would be closer to scientific goals and just as fruitful as PCA, but without detail on those goals that is a matter of speculation.

Note 2: In the case of positive integers, roots and logarithms both have merit as transformations. I note that you state that your data are Poisson distributed without showing any evidence.

| cite | improve this answer | |

As Nick points out, there is some confusion as to whether you mean standardisation (making each variable zero mean and unit variance vectors) or a transformation to make each variable more normally distributed. As you mentioned skewed data, I will address that point.

The square root or log transformations may be applied to the counts to downweight the effects of extreme values, which may dominate the construction of the early axes if doing so would explain large amounts of variance.

For count data though, PCA is rarely competitive in terms of variance explained. Correspondence Analysis will tend to ordinate such data as well if not better than PCA, though it is working on relative compositions (counts) not absolute-valued compositions; in PCA, the following samples with observations on 3 variables

x <- c(1,  5, 1)
y <- c(5, 25, 5)

would be assumed markedly different, but under CA, as these have the same relative composition, would be considered exactly equal.

At this point you should be asking yourself what it is that you hope to achieve with the ordination/dimension reduction and let that lead to you an appropriate method.

| cite | improve this answer | |
  • $\begingroup$ We are in close agreement. But OP didn't profess to using ordination (a term widely understood in ecology, but far less frequently used outside it). $\endgroup$ – Nick Cox Jan 14 '14 at 15:09
  • $\begingroup$ @NickCox As an ecologist I find myself using "ordinate" too freely, although I do confess to finding that idea, ordering things, quite appealing. I didn't notice your note 2 that mentioned "roots" otherwise I may not even have bothered answering. Despite whether the OP wants to ordinate or not; they are by definition doing it with PCA ;-) $\endgroup$ – Gavin Simpson Jan 14 '14 at 15:15
  • 2
    $\begingroup$ Quite so. The only small issue is whether the OP, or anyone else reading the question, is familiar with the term "ordination". As you say, that is ecologists' way of describing what is being done with PCA. Your answer is positive as pointing to correspondence analysis as likely to be a better method. $\endgroup$ – Nick Cox Jan 14 '14 at 15:20

You can use Box-Cox transformation. 𝑓(π‘₯)= (π‘₯^πœ†βˆ’1)/πœ† 𝑖𝑓 πœ†β‰ 0 & π‘™π‘œπ‘”(π‘₯) 𝑖𝑓 πœ†=0 In R the MASS library includes boxcox function to find the optimal πœ†

| cite | improve this answer | |
  • $\begingroup$ The question was about Poisson variables, not a request for a general procedure. $\endgroup$ – mdewey Oct 21 '16 at 8:24
  • 1
    $\begingroup$ @mdewey That doesn't make this answer incorrect. $\endgroup$ – shadowtalker May 18 '17 at 14:36
  • $\begingroup$ @ssdecontrol you are perfectly right but I felt the OP was asking for some more targeted advice. $\endgroup$ – mdewey May 18 '17 at 14:58

Your Answer

By clicking β€œPost Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.