Is it acceptable to reverse a sign of a principal component score? [duplicate]

Question

I have two datasets from similar psycholinguistic experiments. In both of them, information about the participant's reading and spelling ability was collected, then converted into standardized scores zRead and zSpell. The aim is to use these as covariates when examining the priming effects from the experiments. Because these predictors are correlated, and because theoretically it's useful to differentiate their shared variance from their unique effects, I'm looking into using principal components instead, i.e. generate two principal components from the zRead and zSpell scores and use these as the covariates in a linear mixed model.

The problem is that although the principal components come out similarly in each experiment, with PC1 indexing the shared variance and PC2 differentiating reading and spelling ability, the sign of PC2 is different in each dataset, i.e.:

Experiment 1 correlation table:

       zSpell zRead  PC1a   PC2a
zSpell  1.000 0.504 0.867 -0.498
zRead   0.504 1.000 0.867  0.498
PC1a    0.867 0.867 1.000  0.000
PC2a   -0.498 0.498 0.000  1.000

Experiment 2:

       zSpell  zRead  PC1a   PC2a
zSpell  1.000  0.485 0.862  0.508
zRead   0.485  1.000 0.862 -0.508
PC1a    0.862  0.862 1.000  0.000
PC2a    0.508 -0.508 0.000  1.000

The aim is to present both of the related datasets together, so explaining how PC2 represents a (slightly) different thing in each dataset might be confusing. Is it acceptable to reverse score individual components by multiplying by -1? If not, is it okay to reverse all the components at once if the interpretation would make more sense? I can't see this changing the correlational structure of the variables, but I'm not sure if there other reasons to avoid it.

Answer 1

The signs of the eigenvectors are essentially arbitrary; if a colleague were to run the same analyses on the same data but on a different computer it would not be surprising to see one or both eigenvectors (your PC1a, & PC2a) to have different signs. Computing the PCA using the same data on the same computer but via different software packages can also have the same effect.

As such you can quite happily change the sign of the eigenvectors without altering the PCA.

Answer 2

Multiplying a principal component vector by a negative sign is fine. For 0 mean data matrix $X$, PCA calculates,

$$\max_{w} (Xw)^{T}(Xw) $$ s.t. $w^{T}w = 1. $ The equality constraint is there else we could make $||w||$ as large as we wish, so the whole maximization program would be ill-posed. As you can see, if we define $w' = -w$, the new program is identical:

$$\max_{w'} (Xw')^{T}(Xw') $$ s.t. $w'^{T}w' = 1. $.

Answer 3

You could use theory and the PCA to construct your components using a formula.

For example

• Overall Ability = zSpell + zRead
• Relative Aptitude in Spelling in comparison to reading = zSpell - zRead

That's basically what the PCA is doing. However, it removes issues of different signs across analyses.

If the studies use exactly the same variables, you could even take it one step further and standardise the variables in both studies by a common mean and standard deviation. This would make the absolute values in the two studies comparable.

Answer 4

Reversing the sign of the component is fine. The direction of the component is arbitrary. You can check Harman (1976) Modern Factor Analysis as a reference.