Is it acceptable to reverse a sign of a principal component score? [duplicate]

I have two datasets from similar psycholinguistic experiments. In both of them, information about the participant's reading and spelling ability was collected, then converted into standardized scores zRead and zSpell. The aim is to use these as covariates when examining the priming effects from the experiments. Because these predictors are correlated, and because theoretically it's useful to differentiate their shared variance from their unique effects, I'm looking into using principal components instead, i.e. generate two principal components from the zRead and zSpell scores and use these as the covariates in a linear mixed model.

The problem is that although the principal components come out similarly in each experiment, with PC1 indexing the shared variance and PC2 differentiating reading and spelling ability, the sign of PC2 is different in each dataset, i.e.:

Experiment 1 correlation table:

zSpell zRead  PC1a   PC2a
zSpell  1.000 0.504 0.867 -0.498
zRead   0.504 1.000 0.867  0.498
PC1a    0.867 0.867 1.000  0.000
PC2a   -0.498 0.498 0.000  1.000

Experiment 2:

zSpell  zRead  PC1a   PC2a
zSpell  1.000  0.485 0.862  0.508
zRead   0.485  1.000 0.862 -0.508
PC1a    0.862  0.862 1.000  0.000
PC2a    0.508 -0.508 0.000  1.000

The aim is to present both of the related datasets together, so explaining how PC2 represents a (slightly) different thing in each dataset might be confusing. Is it acceptable to reverse score individual components by multiplying by -1? If not, is it okay to reverse all the components at once if the interpretation would make more sense? I can't see this changing the correlational structure of the variables, but I'm not sure if there other reasons to avoid it.

marked as duplicate by amoeba, Peter Flom♦Jan 16 '15 at 10:47

The signs of the eigenvectors are essentially arbitrary; if a colleague were to run the same analyses on the same data but on a different computer it would not be surprising to see one or both eigenvectors (your PC1a, & PC2a) to have different signs. Computing the PCA using the same data on the same computer but via different software packages can also have the same effect.

As such you can quite happily change the sign of the eigenvectors without altering the PCA.

Multiplying a principal component vector by a negative sign is fine. For 0 mean data matrix $X$, PCA calculates,

$$\max_{w} (Xw)^{T}(Xw)$$ s.t. $w^{T}w = 1.$ The equality constraint is there else we could make $||w||$ as large as we wish, so the whole maximization program would be ill-posed. As you can see, if we define $w' = -w$, the new program is identical:

$$\max_{w'} (Xw')^{T}(Xw')$$ s.t. $w'^{T}w' = 1.$.

• Isn't it a bit strange? I mean, isn't ${w}^{T} {X}^{T} X w$ a convex function with respect to $w$ since ${X}^{T} X$ is PSD? Hence how come we try to maximize it? – Royi Dec 24 '16 at 10:23
• @royi, I'm not sure I understand your idea. Can you explain a bit further? – Cam.Davidson.Pilon Dec 24 '16 at 15:48

You could use theory and the PCA to construct your components using a formula.

For example

• Overall Ability = zSpell + zRead
• Relative Aptitude in Spelling in comparison to reading = zSpell - zRead

That's basically what the PCA is doing. However, it removes issues of different signs across analyses.

If the studies use exactly the same variables, you could even take it one step further and standardise the variables in both studies by a common mean and standard deviation. This would make the absolute values in the two studies comparable.

• Thanks. I'm looking into generating the PC's using the formula now. One of the datasets was inherited from the last RA, so I'm not sure if the zRead and zSpell scores were calculated in the exact same way. I'll do a bit of digging to see if the common mean and SD would apply. – Marius Jun 13 '12 at 1:59

Reversing the sign of the component is fine. The direction of the component is arbitrary. You can check Harman (1976) Modern Factor Analysis as a reference.

• Good evening everyone, I need to know which page in MODERN FACTOR ANALYSIS, Harman wrote that changing sign is common. I have researched this problem and until now I not found answers. – user35525 Dec 1 '13 at 3:35